Comment 3 for bug 1659648

Documenting my brainstorm list of things to check next time here:

This might not be perfect solution, but a start.
It is just what came to my mind and might be incomplete.

0. always report the /var/log/libvirt/qemu/guestname.log so one can consider the timing
0. also include the guest xml description (virsh dumpxml <guestname>
0. the host might have extra info in dmesg if it was a host issue
1. is KVM on this system still doing something, check
    perf kvm stat live
2. I'd check if it is hanging in the kernel or the guest. A check of like "ps axlf" will help you.
    If a wchan is assigned and not chainging it is likely hanging on some host kernel queue.
    In that case report that and then try to dump the host so that one can try to crash analyze it.
    If possible to provide a login to the system while in the state - even better.
3. the qemu monitor can sometimes help to check more of the status qemu thinks things are in.
    See https://en.wikibooks.org/wiki/QEMU/Monitor#info for a start
4. if the Host thinks all is fine, but the guest is hanging (virsh shutdown requires the guest to
    cooperate, which might be your case of not stopping it) you might want to debug the guest.
4a.Start with something as trivial as "virsh console". That can't time-warp back, so if it is stuck it
    is empty now - if you happened to set it up to have the main serial console to file you are lucky and can check that.
4b. The next level is guest debugging via gdb. See https://en.wikibooks.org/wiki/QEMU/Monitor#gdbserver

In general the Monitor can do a lot, but it depends so much on the case that I can't write all up.
I hope the links I added help.