image files in _base should not be world-readable

[David Ripton]

Already public in https://bugzilla.redhat.com/show_bug.cgi?id=896085 , so probably no point making this private. But I checked the security vulnerability box anyway so someone else can decide.

We create image files in /var/lib/nova/instances/_base with default permissions, usually 644. It would be better to not make the image files world-readable, in case they contain private data.

[David Ripton (dripton)]

Er, that should have been https://bugzilla.redhat.com/show_bug.cgi?id=893100 Sorry for the mixup.

[Thierry Carrez (ttx)]

Yeah, that should be fixed, but not worth an advisory (as it relies on another vulnerability to be exploited). Can be fixed publicly.

[Matthew Thode (prometheanfire)]

Any update on this, is it on hold til Havana?

[David Ripton (dripton)]

devstack-gate isn't letting this fix in. It's probably a devstack problem, as the patch is very simple, but until I figure it out, it can't go in. If I can get it passing in the next few days, I'd still like to get it into Grizzly. If not, it'll have to slip to Havana.

[Matthew Thode (prometheanfire)]

What is the status on this?

[David Ripton (dripton)]

I'm still working on it, intermittently. The devstack exercise.sh tests fail to start an instance with my patch in, for an unknown reason.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/32146

[Xavier Queralt (xqueralt-deactivatedaccount)]

I don't see a clear solution for this problem in nova and I think this could be better handled in the packaging.

When changing the mode of the instances' directory to 0760 we are also preventing the user 'qemu' to access the images and other files we store there (See nova-compute logs @ 2013-06-07 15:35:00.955 [1]).

From libvirt's documentation [2]:

"The directories /var/run/libvirt/qemu/, /var/lib/libvirt/qemu/ and /var/cache/libvirt/qemu/ must all have their ownership set to match the user / group ID that QEMU guests will be run as. If the vendor has set a non-root user/group for the QEMU driver at build time, the permissions should be set automatically at install time. If a host administrator customizes user/group in /etc/libvirt/qemu.conf, they will need to manually set the ownership on these directories."

In Fedora and RedHat the QEMU guests run as qemu (group qemu) while in debian and ubuntu they runs as libvirt-qemu (group kvm).

An easy solution would be to just change the group of the instances directory to the one qemu is going to use (either qemu or kvm) while still changing the permissions on that directory to 0760. And I'd definitely do this on the packaging level.

Because, besides libvirt, is there any other virt driver storing images in the instances directory?

[1] http://logs.openstack.org/32146/2/check/gate-tempest-devstack-vm-full/21468/logs/screen-n-cpu.txt.gz
[2] http://libvirt.org/drvqemu.html#securitydriver

[David Ripton (dripton)]

Thanks Xavier. My patch failed because it narrowed permissions to only the openstack user, not also the qemu user.

I agree that group permissions should fix this. But I think it's safer to do it internally in nova rather than punting to packagers, if we can. That way we fix it once rather than relying on others to fix it multiple times. The challenge is knowing the correct group in a portable way.

[Matthew Thode (prometheanfire)]

This planned for backport to 2012.2 and/or 2013.1?

[Thierry Carrez (ttx)]

No, but depending on the final shape of the fix it might be possible...

[Joe Gordon (jogo)]

This cannot be in progress with no one assigned, moving back to new

[Thomas Goirand (thomas-goirand)]

IMHO 2 things should be fixed here:

- the /var/lib/nova/instances/_base containing folder should *not* have the world bit x, because otherwise anyone with a login on the system can list files in the folder.

- the images in the folder shouldn't be world readable.

A patch to fix this issue should address both.

Both are of IMO low importance security issues. Low importance because there's a very narrow use case for using a computer for both multi-user system accounts and running a nova compute load. Though narrow, having OpenStack used instead of something like Virtualbox is still a possibility we shouldn't discard, so it shall be fixed ASAP.

As explained on IRC, yes, distributions could potentially address the issue for the folder's rights. Though it's IMO preferable to not off-load this kind of things to downstream. Distributions typically would only create /var/lib/nova, and nothing else.

Also, in Neutron, I've set the rights for /var/lib/neutron to:
drwxr-x---

Is it the view of the project that I should do the same for Nova and everything else? It is my understanding that by doing so, a lot of things would break. Already, in Neutron, this breaks dnsmasq unless dhcp.py is patched to add --user=neutron (which I think is preferable than leaving the folder as world readable). Thoughts welcome.

[Matt Joyce (matt-nycresistor)]

I guess the question then is, is OpenStack requiring specific users and groups to exist on the OS to ensure that this works?

We'll need to know the name of the qemu user and the openstack user ( defined in conf is fine ), but we'll also need to avoid conflicting existing users that could lead to hazards.

We'll also need a shared group for folks accessing this directory path. What should that group be called? Again this falls into does this become a requirement of running openstack?

Might be a question relevant to defcore... but either way, while I agree it would be VERY nice to have this handled in openstack, I fear the potential for conflicting existing distribution decisions in name space.

Compromise Approach:

We allow for conf based definitions of qemu user, openstack user, and new _base access group. But we verify what we can to ensure we don't conflict existing groups and users. ( this may be difficul in the case of the qemu user ).

Thoughts?

[Sean Dague (sdague)]

Honestly, if you have shell users on your compute... things are all kinds of bad.

[Robert Clark (robert-clark)]

@sean - yes that's true but there are all sorts of file disclosure vulnerabilities, leaks and remote reads that don't require shell.

[Tristan Cacqueray (tristan-cacqueray)]

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

I agree with Robert, this expose OpenStack user instance data to all context running on the compute node. Shell users aside, I fail to see why would apache or even the nobody user be allowed to list and read disk files.

[Jeremy Stanley (fungi)]

The way I would usually expect distributions to try and solve situations like this is to define a common openstack-images system group, add the nova and qemu users to it, make the containing directory setgid owned by nova with group openstack-images, and set an appropriately strict umask when calling nova so that it creates group-readable but non-world-readable files. As long as there are ways to convince nova to obey setgid and umask (assuming it doesn't already), this should be doable, right?

[Tristan Cacqueray (tristan-cacqueray)]

The /var/lib/nova/instances directory is likely to be a packaging issue, I don't know how disk image mode bits are set, but at least the disk info is explicitly written as 644 by nova/virt/libvirt/imagebackend.py.

Anyway I closed the OSSA task since multi-user system is not a realistic threat for openstack system.

[Rodney Beede (business2008+launchpad)]

This would be a good hardening opportunity. One use case is you may have unprivileged user accounts that are used for services like monitoring or OS backups unrelated to the OpenStack images themselves. Especially for monitoring these accounts may have basic remote login capability.

Not allowing the unprivileged accounts access via the world read/x bits would be useful.

So perhaps a hardening option that ideally the code would follow a more secure UMASK as well.