I had the very same bug a few days ago with the kvm84 backport for hardy:
Jul 4 07:54:38 toulouse kernel: [4540400.485118] kvm:
segfault at 3e9d20a9 eip 080bcbd8 esp bf9a8774 error 4
It was happening after a while under heavy network load with virtio. The latest version of the package on the ubuntu-virt ppa seems to have fixed it - no segfault since, albeit loads and loads of traffic I made to test it.