Libvirt changes ownership (user/group) of my ISO images

[Jeff Lane ]

Libvirt should NOT be modifying ownership of my ISO images when mounting them to install a KVM instance. This causes issues later when I use zsync to update the images from the latest bits at cdimages.u.c. More importantly, my files should not arbitrarily be stolen from me when they're used BY ME. Period.

-rw------- 1 bladernr bladernr 813694976 Apr 11 08:14 bionic-live-server-amd64.iso
-rw------- 1 libvirt-qemu kvm 754974720 Jul 17 07:09 bionic-server-amd64.iso

One of these I used to install a KVM instance. Now when I run my zsync scripts to update the ISOs, zsync cannot update bionic-server-amd64.iso because I no longer own MY OWN FILE.

I'm filing this because the other bug I found for this is nearly 10 years old and apparently still not fixed.

And I don't agree that the workaround of making libvert use root/root as the user/group is appropriate either.

put simply, when I use my own ISO files to deploy my own KVM instances as my own user, my files should NOT be taken away from me and given to a system user account without me expressly opting into that behaviour.

[Jeff Lane  (bladernr)]

Looking in /etc/libvirt/qemu.conf, this setting would fix this... why is the default not 0 here:

# Whether libvirt should dynamically change file ownership
# to match the configured user/group above. Defaults to 1.
# Set to 0 to disable file ownership changes.
#dynamic_ownership = 1

[Christian Ehrhardt  (paelzer)]

Hi Jeff,
this contains multiple questions at one.

1. for dynamic_ownership "why is the default not 0 here"
=> Because that is the upstream default and actually fixing issues with image permissions for a lot of people.

2. "arbitrarily be stolen from me when they're used BY ME"
This isn't 100% applicable IMHO.
They are used by qemu which for isolation purpose runs under different user/group.
So it is stolen from "you" to be used by qemu under these special constraints.
Also IIRC after the guest is switched off the ownership is restored to what it was before, so it is only changed for currently in use images so that they are actually usable.

All of this is considered system/service owned and not user-owned anyway.
The default is that way to serve most peoples need to not run into trouble with it.
The config exists for more experienced users like you that want to achieve something more special and need the feature disabled.

You can "use it by yourself" with a different setup:
$ qemu-img create -f qcow2 test1.img 10M
XMl like
<domain type='kvm'>
  <name>test</name>
  <memory unit='KiB'>524288</memory>
  <currentMemory unit='KiB'>524288</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <os>
    <type arch='x86_64' machine='pc-i440fx-bionic'>hvm</type>
    <boot dev='hd'/>
  </os>
  <clock offset='utc'/>
  <devices>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/home/paelzer/test1.img'/>
      <target dev='vda' bus='virtio'/>
    </disk>
  </devices>
</domain>

Starting that as a user session will run it in your context
$ virsh -c qemu:///session start test

This will have qemu use:
- your user
- since this matches what the file already has no ownership changes

Session scope has other problems like networking setup as normal user, but that would be matching as "used BY ME" and does not "steal" the files just as expected.

I feel with you but I'd currently not change the default as it would break too many other people and hope my explanations could outline why it is the way it is.
If you are ok with your setup to disable the dynamic ownership then please go on with that.

[Andreas Hasenack (ahasenack)]

I was bitten by this issue several times already. It affects me when I want to use an ISO image I have stored in my NAS server, via nfs for example. Since root is squashed, libvirt fails to change the ownership and thus fails to launch the VM. I have to then copy it over to my laptop and then tell libvirt to use that one, which it can chown/chmod to its heart content.

I thought I had filed a bug about this already, maybe against virt-manager, but couldn't find it.

I'll take a look at the dynamic ownership setting.

[Andreas Hasenack (ahasenack)]

> Also IIRC after the guest is switched off the ownership is restored to what it was before,
> so it is only changed for currently in use images so that they are actually usable.

That didn't happen to the ISO image I used to bootstrap a new VM.

Aren't we talking about two different things here? The VM image file, and the ISO image used to boot from to start an installation?

[Jeff Lane  (bladernr)]

Thanks Christian.

Sadly, I don't care to create and spin up KVM instances from a command line, I prefer (as I presume most people do) to use virt-manager.

When I set dynamic_ownership = 0, I am unable to boot due to a permission denied error:

Error starting domain: internal error: process exited while connecting to monitor: 2018-08-09T18:44:58.668207Z qemu-system-x86_64: -drive file=/home/bladernr/Downloads/bionic-live-server-amd64.iso,format=raw,if=none,id=drive-ide0-0-1,readonly=on: Could not open '/home/bladernr/Downloads/bionic-live-server-amd64.iso': Permission denied

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 89, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 125, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 82, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/domain.py", line 1508, in startup
    self._backend.create()
  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 1062, in create
    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirtError: internal error: process exited while connecting to monitor: 2018-08-09T18:44:58.668207Z qemu-system-x86_64: -drive file=/home/bladernr/Downloads/bionic-live-server-amd64.iso,format=raw,if=none,id=drive-ide0-0-1,readonly=on: Could not open '/home/bladernr/Downloads/bionic-live-server-amd64.iso': Permission denied

So if I change it back to 1 and restart libvirtd I can start KVM instances, but we're back to the original issue.

Is there some other setting necessary so virt-manager (running via my normal user) doesn't try launching things as the libvirt-kvm user, which is what I presume is the root of all this?

bladernr 15974 2.4 0.3 1269564 118660 ? Ssl 14:52 0:01 \_ /usr/bin/python2 /usr/share/virt-manager/virt-manager
root 5898 0.0 0.0 211108 7480 ? Ssl 13:11 0:00 /usr/sbin/virtlogd
root 15674 0.4 0.1 1894392 47552 ? Ssl 14:51 0:00 /usr/sbin/libvirtd

Or... also possible, am I just confused?

[Christian Ehrhardt  (paelzer)]

Hi Jeff,
what I tried to outline before for virsh applies to virt-manager as well as they are all just libvirt frontends.

So the default of qemu:///system will be a system owned guest with all the implications (good and bad) that come with it.

If you want to run "as user" it should be qemu:///session as shown above.
The same works for virt-manager.
You can go to "File -> Add connection" and enter "qemu:///session" as custom URL.
With a guest started like that I have:
- running qemu-system-x86_64 as my UID
  F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
  2 1000 11683 1 20 0 1856652 333760 poll_s Sl ? 3:51 qemu-sys...
- my disk image still self-owned
  -rw------- 1 paelzer paelzer 16108814336 Aug 10 07:34
  /home/paelzer/.local/share/libvirt/images/qemu-session-18.04.qcow2

As I mentioned before user session comes OTOH with other implications.
The one I heard most about are issues with networking that needs different setup and some help to get working depending on the network setup.

My default case - install from 18.04 ISO in virt-manager with qemu:///session that I just tried worked for me (including network to external sites).

[Launchpad Janitor (janitor)]

[Expired for libvirt (Ubuntu) because there has been no activity for 60 days.]

[Heather Ellsworth (hellsworth)]

I am experiencing this issue as well since installing lunar (fresh install). I've installed virt-manager ant it seemed to setup a default network (virbr0) fine, connect to it fine (as seen in the Connection Details -> Virtual Networks tab). However when I go to create my first VM On this fresh install, I see "Internal error: process exited while connecting to monitor" (see screenshot for complete error). I cancel the installation and see that the iso has remained in the state where its owner is "libvirt-qemu" and its group is "kvm", rather than setting it back to what it was before virt-manager changed the permissions.

I went to their IRC for help, and found out that if you move the iso and change the permissions, VM creation works now. Thanks @laine!

  $ mv ~/Downloads/ubuntu-22.10-desktop-amd64.iso /var/lib/libvirt/images/
  $ sudo chown root:root /var/lib/libvirt/images/ubuntu-22.10-desktop-amd64.iso

Then create your VM with the iso and it works.

Perhaps /var/lib/libvirt/images is the magic directory because the "default" media volume has location "/var/lib/libvirt/images". I always add a "Downloads" media volume and use this, but perhaps the file access from this added volume is faulty for some reason.

Still, moving the iso and changing its ownership is a *workaround*. An iso located in the user's Downloads folder (or anywhere in $HOME) should be usable by virt-manager. Therefore I reopened the bug.

[Heather Ellsworth (hellsworth)]

Here is a video showing the problem.

[Christian Ehrhardt  (paelzer)]

Hi Heather,
I assume this is mixing a few problems, and in doing so we got stuck in the past I think.

- One is the change of ownership, wich was sufficiently discussed here as while some would like it different is a tunable and at the default that is better and helpful for the majority (the rest can tune it to be different, but very often it then won't work as guests run in isolated apparmor and user permissions)
- Another aspect is that usually image ownership is restored back after the guest stops.
  But that does not (reliably) seem to happen for .iso files which it should.
  => bug 2002771
- The other one is the use-case of "download ISO to ~/Downloads and using it from there" would be nice to work
  => bug 2002772

The latter two are potential real issue which we should look at, but are independent to this bug reported here. Splitting them up will help to not drown in the "what should be the default" discussion and focus on what we could change without that burden.
I have forked off two bugs as referenced above to have an undiluted new view at them.

Please subscribe and chime in there for the sub-problems you are interested in.

[Heather Ellsworth (hellsworth)]

Christian, Thank you so much. You are right that there are two issues and we should focus the discussion in the two issues (although the second issue is actually bug 2002773). I will set the status to Invalid.

[mohdforever (mohdforever)]

the easiest way is to add your user in config file as below

echo "
user = "$USER"
group = "$USER"
dynamic_ownership = 1
remember_owner = 1
swtpm_user = "swtpm"
swtpm_group = "swtpm"
" | sudo tee -a /etc/libvirt/qemu.conf

[Christian Ehrhardt  (paelzer)]

Hi Mohd,
most of that is the default and the rest could IMHO cause more trouble than help.
In detail:
1. dynamic_ownership, remember_owner, swtpm_user, swtpm_group are already by default on the values you suggested.
2. setting user/owner changes the user/group sets these for the QEMU processes run by system instances. This is by default libvirt-qemu:kvm, among others reasons one of them is to ensure a guest that might be exploited only directly reaching permissions/capabilities that are not too wide reaching.

As we have split before, this actually consists of two bugs.
1. using an iso file as downloaded should work in a guest (it does) - bug 2002773
2. restoring file ownership after execution isn't always working - bug 2002771

Yes, you might mitigate the second bug by just executing as the user you want your files to be owned. Because then if they fail to restore back they will be what you want, but as outlined above there are concerns in regard to security - and on a multi user system a single user/group there also isn't very helpful.

Just recently we were able to understand in bug 2002771 why this sometimes worked and sometimes did not (depends on use case and how things are attached/detached) which eluded further debugging for a long time.