libvirt sometimes hangs when using pulseaudio

[Jamie Strandboge]

Occasionally, a virtual machine will hang during boot when started from virt-manager. This seems to be racy or related to some interaction with pulseaudio. I saw this when using an 8.10 and 9.10 desktop livecd. During the boot process, the VM would hang and all of libvirt would be unresponsive (ie 'virsh list' would also hang). This happens here maybe 1 out of 10 times.

Until bug #453329 is fixed, you will have to disable apparmor for libvirt with:
$ sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd
$ sudo /etc/init.d/libvirt-bin restart

This is because that bug masks the problem by denying access to pulseaudio, and therefore audio is not available in the guest and the boot proceeds like normal. I will try to obtain an 'strace -p' of the kvm process when I see another hang. I have witnessed 4 hangs in maybe 20 boots (3 in a row, a couple successful boots and 1 outlier).

ProblemType: Bug
Architecture: amd64
Date: Fri Oct 16 15:07:59 2009
DistroRelease: Ubuntu 9.10
LiveMediaBuild: Ubuntu 9.04 "Jaunty Jackalope" - Release amd64 (20090420.1)
Package: libvirt-bin 0.7.0-1ubuntu11
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.31-14.47-generic
SourcePackage: libvirt
Uname: Linux 2.6.31-14-generic x86_64

[Jamie Strandboge (jdstrand)]

This has nothing to do with the apparmor plugin. If you disable apparmor it will still hang.

[Chuck Short (zulcss)]

<jdstrand> zul: 453453 is not fixed. apparmor didn't cause the problem. fixing #453329 only makes #453453 noticeable. I talked to kirkland about it at length on friday. he said he was able to reproduce it, though I don't recall how
<zul> k
 so im gonig to set it to confirmed
<jdstrand> zul: that sounds fine. I invalidated the apparmor task
<zul> thanks for the heads up

[Dustin Kirkland  (kirkland)]

Marking won't fix for karmic.

If someone has or finds a good patch, this could be a candidate for an SRU.

:-Dustin

[Daniel Nelson (packetcollision)]

This affects me on 2 computers, 100% of the time, and finding information about the problem was quite hard. If this isn't going to be fixed, perhaps a warning that having a soundcard in the VM may cause libvirt to hang would be a good idea.

Interestingly, the exact same VM booted directly with the exact same kvm line as the one libvirt uses (as pulled from ps -ef |grep kvm) works fine.

I haven't yet been able to test this extensively, but it appears that removing the sound card from the VM is indeed a workaround.

[Jamie Strandboge (jdstrand)]

One idea might be to not add the soundcard when people create VMs using virt-manager. This of cource doesn't address existing VMs....

[Dustin Kirkland  (kirkland)]

I'm moving this to priority Critical. This is happening to me every time. It makes virt-manager basically unusable.

:-Dustin

[Dustin Kirkland  (kirkland)]

So I've added a bunch of packages to this bug, as I'm not entirely sure where it's at.

Most likely, most of these will be marked Invalid eventually.

But at this point, the bug is somewhere among:
 * virt-manager
 * libvirt
 * qemu-kvm
 * pulseaudio.

I have noticed that if I stop the pulseaudio process on my host, I do not see the problem.

Also, if I run qemu-kvm directly with `export QEMU_AUDIO_DRV=alsa`, I do not see the problem.

:-Dustin

[Daniel T Chen (crimsun)]

If using the alsa backend for qemu makes the symptom go away, it's fairly unlikely that PulseAudio is the culprit, since using the alsa backend still routes through PA on the host due to the pulse alsa-plugin being used.

It might make sense to ship Karmic with that env var (QEMU_AUDIO_DRV=alsa)...

[Daniel Nelson (packetcollision)]

I'd be happy to test the env var solution. What file should I add it to?

[Jamie Strandboge (jdstrand)]

These links were recently pointed out to me, and directly speak to this bug:
http://osdir.com/ml/libvir-list/2009-09/msg00494.html
https://bugzilla.redhat.com/show_bug.cgi?id=486112
http://fedoraproject.org/wiki/Features/VirtVNCResourceTunnel

[Soren Hansen (soren)]

I haven't been able to reproduce it, unfortunately. I understand the problem, though.

Simply put, the problem is mostly that since the VMs run as root rather than your user, it's not completely straight forward how to deal with access to the sound device.

The real fix would be to tunnel sound through VNC. However, gtk-vnc and virt-manager do not support this yet (kvm does).

We could hack libvirt to ignore audio devices for VM's on qemu:///system. That would certainly fix this bug, but would remove sound support for people for whom it actually works.

Making virtinst not add sound devices will not fix existing VM's.

Suggestions?

[Luke Yelavich (themuso)]

The only way that I can think of solving this for now, is to run PulseAudio at the system level, which however brings its own caviots and issues. Alternatively, we could investivate ulseAudio's TCP communication module, and set kvm up to communicate with pulse via tcp/UDP. I know very little about tat module, but I believe this is what it is for.

[Soren Hansen (soren)]

Perhaps I should clarify that I'm looking for an SRUable solution. For
Lucid I'm somewhat hopeful we'll tunnel sound over VNC and have a means
for hooking that up through gtk-vnc(/virt-manager).

[Brian Rogers (brian-rogers)]

If virt-manager had to push out audio through pulseaudio, it'd have to handle having no pulseaudio running when it starts, and it'd have to transfer audio output to the correct pulseaudio on demand and correctly handle the daemon being terminated.

What about doing it the other way, pull instead of push? It might be possible for virt-manager to publish an audio stream locally that pulseaudio can then tune into.

[Dustin Kirkland  (kirkland)]

Marking this won't fix for Karmic, adding Lucid task.

Can anyone reproduce this behavior in Lucid?

[Dustin Kirkland  (kirkland)]

Marking invalid against qemu-kvm. I really don't think the bug is in qemu-kvm. Rather, it must be somewhere among libvirt and pulseaudio.

I have verified that sound works very well in kvm in Lucid and pulseaudio (without libvirt).

I have also noticed that virt-manager in Lucid does not add a sound device by default when creating a virtual machine (you can add one after the fact). I have not gotten guest sound to work correctly under Lucid's virt-manager, though.

[Daniel T Chen (crimsun)]

Need a verbose log (https://wiki.ubuntu.com/PulseAudio/Log) for the PA task when this occurs

[Jamie Strandboge (jdstrand)]

I haven't seen this for months on Lucid. I am going to mark the Lucid tasks as Fix Released and reopen them if I happen to see it.

[Dustin Kirkland  (kirkland)]

Agreed, Jamie. Sound has been working marvelously for me in Lucid!