nova interface attach command fails in 14.04 with 2014.1.1: error: "libvirtError: internal error: unable to execute QEMU command 'getfd': No file descriptor supplied via SCM_RIGHTS"

[Praveen Yalagandula]

On 14.04 ubuntu with icehouse bug fix release (2014.1.1), nova "interface-attach" command is failing (used to work well in previous versions), while trying to attach an interface to a live VM.

Here is the exception in the nova-compute.log file:
2014-07-25 13:55:07.762 8546 ERROR nova.virt.libvirt.driver [req-e55e6443-9067-44fd-b0bc-21d102307c2a 03a68cb6d8354bbbae9011bacacea4f8 fde333f66694463ea8f2ff994d97998a] [instance: 759578c6-0d08-4bb9-b571-686296d07059] attaching network adapter failed.
2014-07-25 13:55:08.161 8546 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: <nova.objects.instance.Instance object at 0x7fe48cfcce50>
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher incoming.message))
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 393, in decorated_function
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4344, in attach_interface
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher self.driver.attach_interface(instance, image_meta, network_info[0])
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1423, in attach_interface
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher raise exception.InterfaceAttachFailed(instance)
2014-07-25 13:55:08.161 8546 TRACE oslo.messaging.rpc.dispatcher InterfaceAttachFailed: <nova.objects.instance.Instance object at 0x7fe48cfcce50>

I modified the nova/virt/libvirt/driver.py to add a LOG.exception() to track where and why the interface failed and I saw the following exception:

2014-07-25 13:56:23.878 7143 ERROR nova.virt.libvirt.driver [req-aa938b8a-f31d-4b85-8d29-291bc3acd2ba 03a68cb6d8354bbbae9011bacacea4f8 fde333f66694463ea8f2ff994d97998a] attach failed
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver Traceback (most recent call last):
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1418, in attach_interface
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver virt_dom.attachDeviceFlags(cfg.to_xml(), flags)
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver rv = execute(f,*args,**kwargs)
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver rv = meth(*args,**kwargs)
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/dist-packages/libvirt.py", line 513, in attachDeviceFlags
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2014-07-25 13:56:23.878 7143 TRACE nova.virt.libvirt.driver libvirtError: internal error: unable to execute QEMU command 'getfd': No file descriptor supplied via SCM_RIGHTS

Compute node setup:

1) Release of ubuntu:
Description: Ubuntu 14.04 LTS
Release: 14.04

2) dpkg --list | grep qemu
ii ipxe-qemu 1.0.0+git-20131111.c3d1e78-2ubuntu1 all PXE boot firmware - ROM images for qemu
ii qemu-keymaps 2.0.0+dfsg-2ubuntu1.1 all QEMU keyboard maps
ii qemu-kvm 2.0.0+dfsg-2ubuntu1.1 amd64 QEMU Full virtualization on x86 hardware (transitional package)
ii qemu-system 2.0.0+dfsg-2ubuntu1.1 amd64 QEMU full system emulation binaries
ii qemu-system-arm 2.0.0+dfsg-2ubuntu1.1 amd64 QEMU full system emulation binaries (arm)
ii qemu-system-common 2.0.0+dfsg-2ubuntu1.1 amd64 QEMU full system emulation binaries (common files)
ii qemu-system-mips 2.0.0+dfsg-2ubuntu1.1 amd64 QEMU full system emulation binaries (mips)
ii qemu-system-misc 2.0.0+dfsg-2ubuntu1.1 amd64 QEMU full system emulation binaries (miscelaneous)
ii qemu-system-ppc 2.0.0+dfsg-2ubuntu1.1 amd64 QEMU full system emulation binaries (ppc)
ii qemu-system-sparc 2.0.0+dfsg-2ubuntu1.1 amd64 QEMU full system emulation binaries (sparc)
ii qemu-system-x86 2.0.0+dfsg-2ubuntu1.1 amd64 QEMU full system emulation binaries (x86)
ii qemu-utils 2.0.0+dfsg-2ubuntu1.1 amd64 QEMU utilities

dpkg --list | grep nova
ii nova-common 1:2014.1.1-0ubuntu1 all OpenStack Compute - common files
ii nova-compute 1:2014.1.1-0ubuntu1 all OpenStack Compute - compute node base
ii nova-compute-kvm 1:2014.1.1-0ubuntu1 all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 1:2014.1.1-0ubuntu1 all OpenStack Compute - compute node libvirt support
ii nova-network 1:2014.1.1-0ubuntu1 all OpenStack Compute - Network manager
ii python-nova 1:2014.1.1-0ubuntu1 all OpenStack Compute Python libraries
ii python-novaclient 1:2.17.0-0ubuntu1 all client library for OpenStack Compute API

dpkg --list | grep libvirt
ii libvirt-bin 1.2.2-0ubuntu13.1.1 amd64 programs for the libvirt library
ii libvirt0 1.2.2-0ubuntu13.1.1 amd64 library for interfacing with different virtualization systems
ii nova-compute-libvirt 1:2014.1.1-0ubuntu1 all OpenStack Compute - compute node libvirt support
ii python-libvirt 1.2.2-0ubuntu1 amd64 libvirt Python bindings

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[YAMAMOTO Takashi (yamamoto)]

why official jenkins doesn't suffer from this?
is there any known workaround?
this is one of show-stoppers for our CI (ofagent CI) upgrading to 14.04.