vm fails to be define virsh by the user who is member of group configured in qemu.conf

Bug #1676208 reported by Serguei Bezverkhi on 2017-03-26
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
libvirt (Ubuntu)
Undecided
Unassigned

Bug Description

In the containers environment virsh define run from the host fails. libvirt runs in the ubuntu container and is exposing /var/lib/libvrti as well as as /var/run to the host, so libvirtd sockets are visible on the host.

libvirt run user nova group nova. User root has been added in libvirtd container into nova group. SO it should have permissions to define vm even when it runs from the host with virs command.

2017-03-26 18:16:07.076840 | + sudo virsh define /home/jenkins/workspace/gate-kolla-kubernetes-deploy-ubuntu-source-3-ironic-nv/tests/bin/../conf/ironic/vm-1.xml
2017-03-26 18:16:07.232451 | error: Failed to define domain from /home/jenkins/workspace/gate-kolla-kubernetes-deploy-ubuntu-source-3-ironic-nv/tests/bin/../conf/ironic/vm-1.xml
2017-03-26 18:16:07.232540 | error: internal error: QEMU / QMP failed: 2017-03-26 18:16:07.130+0000: 4815: debug : virFileClose:102 : Closed fd 30
2017-03-26 18:16:07.232570 | 2017-03-26 18:16:07.130+0000: 4815: debug : virFileClose:102 : Closed fd 32
2017-03-26 18:16:07.232608 | 2017-03-26 18:16:07.130+0000: 4815: debug : virFileClose:102 : Closed fd 26
2017-03-26 18:16:07.232649 | 2017-03-26 18:16:07.130+0000: 4815: debug : virExec:736 : Setting child uid:gid to 42436:42436 with caps 0
2017-03-26 18:16:07.232686 | qemu-system-x86_64: could not acquire pid file: Permission denied

Ubuntu 16.04
qemu-system-x86 (1:2.5+dfsg-5ubuntu10.9)
libvirt0:amd64 (1.3.1-1ubuntu10.8)

Attached in the libvirtd.log collected from the libvirtd container.

Serguei Bezverkhi (sbezverk) wrote :

Hi Serguei,
I thank you for your report, but must admit you lost me at the description.

Trying to understand your setup

1. you are running virsh define in a containter but from a host, do you mean like this?
$ lxc exec <containername> -- virsh define <guestname.xml>

2. You are exposing /var/lib/libvirt and /var/run to the host - Don't you usually expose from the Host to the Container. Could you provide the setup you used to do so, like the lxc profile or whatever applies?

3. in your container that runs libvirt, you added UID of root to the group of NOVA?

I guess depending on your answer to #2 a lot of my assumptions might be totally wrong, so please help to clarify those.

4.
From the error I think what happens is that your case is that the libvirt (wherever it is) can't reach qemu for the capability checks on the define. I think there is no need to run a full define to trigger this, could you also report what the following reports in your case (I'd assume it aborts as well):
 $ virth capabilities

5.
And if that fails I don't yet see so much why/where this is about user/group permissions except invoking qemu-system-x86_64 but that is done by lobvirts user. The libvirt config might help to understand. Could you attach the config files that are reported (on your container?) when running:
 $ sudo dpkg --verify libvirt-bin

Changed in libvirt (Ubuntu):
status: New → Incomplete
Serguei Bezverkhi (sbezverk) wrote :

Apologies for creating confusion. It is kind of complicated setup :) It is docker containers environment. Everything runs on the same physical host. libvirtd runs inside of a docker container and is fine (it is part of kola-kubernetes openstack). What I need to do it and the same works with centos, is to define VM manually on the host and then be able to start it. I would like to use the socket and PID of already existing libvirtd process running inside of a docker container. Since libvirtd container runs in the same namespace as the host, both host and container share the same PID namespaces and network namespaces.
I cannot run any virsh commands as every time it fails to connect to the hypervisor. On the next run I will collect dpkg output you requested. Please let me know if you need further clarification.

Serguei Bezverkhi (sbezverk) wrote :
Download full text (27.3 KiB)

2017-03-27 13:28:41.779585 | + sudo virsh capabilities
2017-03-27 13:28:44.447707 | <capabilities>
2017-03-27 13:28:44.447812 |
2017-03-27 13:28:44.447854 | <host>
2017-03-27 13:28:44.447910 | <uuid>df3da4c3-707b-4573-ac97-fceb53dc262b</uuid>
2017-03-27 13:28:44.447948 | <cpu>
2017-03-27 13:28:44.447993 | <arch>x86_64</arch>
2017-03-27 13:28:44.448041 | <model>SandyBridge</model>
2017-03-27 13:28:44.448087 | <vendor>Intel</vendor>
2017-03-27 13:28:44.448141 | <topology sockets='8' cores='1' threads='1'/>
2017-03-27 13:28:44.448190 | <feature name='osxsave'/>
2017-03-27 13:28:44.448240 | <feature name='hypervisor'/>
2017-03-27 13:28:44.448287 | <feature name='xsaveopt'/>
2017-03-27 13:28:44.448335 | <pages unit='KiB' size='4'/>
2017-03-27 13:28:44.448384 | <pages unit='KiB' size='2048'/>
2017-03-27 13:28:44.448424 | </cpu>
2017-03-27 13:28:44.448467 | <power_management/>
2017-03-27 13:28:44.448512 | <migration_features>
2017-03-27 13:28:44.448555 | <live/>
2017-03-27 13:28:44.448598 | <uri_transports>
2017-03-27 13:28:44.448648 | <uri_transport>tcp</uri_transport>
2017-03-27 13:28:44.448698 | <uri_transport>rdma</uri_transport>
2017-03-27 13:28:44.448741 | </uri_transports>
2017-03-27 13:28:44.448786 | </migration_features>
2017-03-27 13:28:44.448826 | <topology>
2017-03-27 13:28:44.448869 | <cells num='1'>
2017-03-27 13:28:44.448913 | <cell id='0'>
2017-03-27 13:28:44.448965 | <memory unit='KiB'>8174408</memory>
2017-03-27 13:28:44.449020 | <pages unit='KiB' size='4'>2043602</pages>
2017-03-27 13:28:44.449074 | <pages unit='KiB' size='2048'>0</pages>
2017-03-27 13:28:44.449117 | <distances>
2017-03-27 13:28:44.449166 | <sibling id='0' value='10'/>
2017-03-27 13:28:44.449211 | </distances>
2017-03-27 13:28:44.449257 | <cpus num='8'>
2017-03-27 13:28:44.449315 | <cpu id='0' socket_id='0' core_id='0' siblings='0'/>
2017-03-27 13:28:44.449376 | <cpu id='1' socket_id='1' core_id='0' siblings='1'/>
2017-03-27 13:28:44.449434 | <cpu id='2' socket_id='2' core_id='0' siblings='2'/>
2017-03-27 13:28:44.449493 | <cpu id='3' socket_id='3' core_id='0' siblings='3'/>
2017-03-27 13:28:44.449554 | <cpu id='4' socket_id='4' core_id='0' siblings='4'/>
2017-03-27 13:28:44.449615 | <cpu id='5' socket_id='5' core_id='0' siblings='5'/>
2017-03-27 13:28:44.449673 | <cpu id='6' socket_id='6' core_id='0' siblings='6'/>
2017-03-27 13:28:44.449733 | <cpu id='7' socket_id='7' core_id='0' siblings='7'/>
2017-03-27 13:28:44.449776 | </cpus>
2017-03-27 13:28:44.449815 | </cell>
2017-03-27 13:28:44.449897 | </cells>
2017-03-27 13:28:44.449940 | </topology>
2017-03-27 13:28:44.449980 | <secmodel>
2017-03-27 13:28:44.450025 | <model>none</model>
2017-03-27 13:28:44.450090 | <doi>0</doi>
2017-03-27 13:28:44.450132 | </secmodel>
2017-03-27 13:28:44.450171 | <secmodel>
2017-03-27 13:28:44.450215 | <model>dac</model>
2017-03-27...

Robie Basak (racb) on 2017-03-28
Changed in libvirt (Ubuntu):
status: Incomplete → New
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers