Volume details shows attached compute host for non-admins

Bug #1740950 reported by Matt Riedemann
258
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
Undecided
Rajat Dhasmana
OpenStack Security Advisory
Won't Fix
Undecided
Unassigned

Bug Description

The Cinder API shows attached attachments with volume details, and it includes the attached_host information for the attachments, if present, regardless of whether or not an admin token is being used:

https://github.com/openstack/cinder/blob/a95c9e5668f6a7596e0198cca2b6b7fef20ab3e9/cinder/api/v2/views/volumes.py#L126

This means a non-admin can see compute host details when a volume is attached to their instance.

This is easily recreatable in devstack:

1. Show that I'm the demo user (can't perform admin things like service-list):

stack@queens:~/devstack$ nova service-list
ERROR (Forbidden): Policy doesn't allow os_compute_api:os-services to be performed. (HTTP 403) (Request-ID: req-9e4c01ef-6bf4-41af-9768-26936f137640)

2. Create a server and a volume and attach the volume to the server:

stack@queens:~/devstack$ nova volume-attach test-server1 373c2a29-f6e7-4931-adda-6b87a193e9f9
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | /dev/vdb |
| id | 373c2a29-f6e7-4931-adda-6b87a193e9f9 |
| serverId | 43bc2379-86d5-49b2-a87c-5351e19ca69c |
| volumeId | 373c2a29-f6e7-4931-adda-6b87a193e9f9 |
+----------+--------------------------------------+
stack@queens:~/devstack$ openstack volume list
+--------------------------------------+------------------+--------+------+---------------------------------------+
| ID | Name | Status | Size | Attached to |
+--------------------------------------+------------------+--------+------+---------------------------------------+
| 373c2a29-f6e7-4931-adda-6b87a193e9f9 | test-vol1 | in-use | 1 | Attached to test-server1 on /dev/vdb |

3. Show the volume details to see the attachment and the attached_host:

stack@queens:~/devstack$ openstack volume show test-vol1
+------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{u'server_id': u'43bc2379-86d5-49b2-a87c-5351e19ca69c', u'attachment_id': u'95dbfdd8-a615-4c66-a32a-79dc0dfd344e', u'attached_at': u'2018-01-02T21:37:57.000000', u'host_name': u'queens', u'volume_id': u'373c2a29-f6e7-4931-adda-6b87a193e9f9', u'device': u'/dev/vdb', u'id': u'373c2a29-f6e7-4931-adda-6b87a193e9f9'}] |

You can see that the host_name is in the response and the value is "queens" which is the compute hostname on this single-node install:

stack@queens:~/devstack$ sudo hostname
queens

This is using LVM by default with devstack. I tried the same thing against vexxhost and host_name came back as None, so it might be backend-specific (vexxhost is using Rbd).

This is the vexxhost output for the same scenario:

(vexxhost) user@ubuntu:~$ openstack volume show test-vol1
+------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{u'server_id': u'd1643bfb-1701-4adc-9980-c2553203d1f8', u'attachment_id': u'3e14b74b-7ee3-4fd8-9b8d-7a86be17cff9', u'attached_at': u'2018-01-02T21:48:32.000000', u'host_name': None, u'volume_id': u'588f0fdf-1531-44e7-8d92-16d4150c3634', u'device': u'/dev/vdb', u'id': u'588f0fdf-1531-44e7-8d92-16d4150c3634'}] |

--

This has been around since Havana: https://review.openstack.org/#/c/34125/

This is significant because the default compute API policy is to not show the compute host/node information for a server instance for non-admin users, but with this exposure in the Cinder API, non-admin users can get around that restriction (potentially - again, it seems to depend on volume backend) to see the compute host on which the volume is attached.

Revision history for this message
Matt Riedemann (mriedem) wrote :

I believe I've found the issue, and it's a regression in the 3.27 Cinder volume attachment_update API:

PUT /volumes/{id}/attachments/{attachment_id}

The volume manager code is updating the volume with the host from the connector dict, which nova gets from os-brick, and setting that on the volume here:

https://github.com/openstack/cinder/blob/a95c9e5668f6a7596e0198cca2b6b7fef20ab3e9/cinder/volume/manager.py#L4370

Nova only started using this code as of this change:

https://review.openstack.org/#/c/330285/

Which is not released yet.

That's also the reason why I couldn't reproduce this on vexxhost since they are running Pike code. And before https://review.openstack.org/#/c/330285/, nova would call the os-attach volume actions API but never passed a host name directly for that, so this was always None:

https://github.com/openstack/cinder/blob/master/cinder/volume/manager.py#L1189

So the question is what to do about this before the Queens release. There are a couple of options:

1. Add a policy rule in Cinder to not expose the attached_host field in the response to non-admins. To not break backward compatibility, you'd likely need to default this to allow admin_or_owner.
2. Don't store the attached_host value when calling attachment_update, and if some client needs to actually set the hostname for the attachment to get it later, like glance, it should use the os-attach volume action API.

Revision history for this message
Tristan Cacqueray (tristan-cacqueray) wrote :

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

Changed in ossa:
status: New → Incomplete
description: updated
Revision history for this message
Jeremy Stanley (fungi) wrote :

Just to clarify, this is only affecting openstack/nova master after change 330285 merged? If so, and it gets fixed before Queens releases, the VMT wouldn't normally issue an advisory as no stable releases are impacted. In that case, it may also make sense to switch to a public workflow so the embargo process doesn't slow down design discussions and patch review.

Revision history for this message
Matt Riedemann (mriedem) wrote :

@Jeremy - correct. It's not yet released (tagged). I reported as a private security bug before I realized the root cause, so I'm OK with making this public so we get visibility and a fix before Queens GA.

Revision history for this message
Jeremy Stanley (fungi) wrote :

In that case, I'll triage it as a Class Y ("only found in development release") report from the VMT's perspective. If it lingers past Queens release day, we can revisit the advisory task.

https://security.openstack.org/vmt-process.html#incident-report-taxonomy

Changed in ossa:
status: Incomplete → Won't Fix
information type: Private Security → Public Security
Revision history for this message
Matt Riedemann (mriedem) wrote :

Discussed this at the Rocky PTG, agreed to add a policy rule to cinder to only show host_name if using an admin-scoped token. The cinder team needs to talk to the glance team about any impacts this might have on cinder-backed glance storage.

Matt Riedemann (mriedem)
Changed in cinder:
status: New → Triaged
Revision history for this message
Matt Riedemann (mriedem) wrote :

Note that I can also call GET /v3/{project_id}/attachments/{attachment_id} on any volume attachment to my server as a non-admin and get the connection_info dict details about the host and volume backend that the volume is attached to.

There is no policy rule for that API in Cinder as of Ussuri:

https://opendev.org/openstack/cinder/src/tag/15.0.0/cinder/api/v3/attachments.py#L56

https://opendev.org/openstack/cinder/src/tag/15.0.0/cinder/api/v3/views/attachments.py#L35

That's likely because it's an admin_or_owner type implied policy on just getting the attachment and the connection_info dict in the attachment resource is how nova can get the connection_info like nova used to use the os-initialize_connection API:

https://docs.openstack.org/api-ref/block-storage/v3/?expanded=initialize-volume-attachment-detail#initialize-volume-attachment

The response of which is the connection_info:

https://opendev.org/openstack/cinder/src/tag/15.0.0/cinder/api/contrib/volume_actions.py#L177

Jeremy Stanley (fungi)
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.opendev.org/713231

Changed in cinder:
assignee: nobody → Rajat Dhasmana (whoami-rajat)
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.opendev.org/726751

Revision history for this message
Brian Rosmaita (brian-rosmaita) wrote :

Note: the Attachment API issue is being tracked by https://launchpad.net/bugs/1736773

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.opendev.org/726751
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=9d5c340130ac056e7558468cc61872db65140c9b
Submitter: Zuul
Branch: master

commit 9d5c340130ac056e7558468cc61872db65140c9b
Author: Rajat Dhasmana <email address hidden>
Date: Mon May 11 08:21:09 2020 +0000

    Don't show host info to non-admins

    While getting volume details, if the user is a non-admin then
    hide the host name.

    Change-Id: Iaf0ac52d9227f9a0efbf32b1faca78c8456a84ca
    Closes-Bug: #1740950

Changed in cinder:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public Security information  
Everyone can see this security related information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.