default paste_deploy.flavor is none, but config file text implies it is 'keystone' (was: non-admin users can see all tenants' images even when image is private)

Bug #1799588 reported by Jonathan Mills on 2018-10-23
274
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Glance
Medium
Michael Moore
OpenStack Security Advisory
Undecided
Unassigned

Bug Description

[root@vm013 glance]# cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)
[root@vm013 glance]# rpm -qa |grep glance |sort
openstack-glance-16.0.1-1.el7.noarch
openstack-glance-doc-16.0.1-1.el7.noarch
python2-glanceclient-2.10.0-1.el7.noarch
python2-glance-store-0.23.0-1.el7.noarch
python-glance-16.0.1-1.el7.noarch
python-glanceclient-doc-2.10.0-1.el7.noarch
[root@vm013 glance]# md5sum /etc/glance/policy.json
a4f29d0f75bbc04f1d83a1abdf0fda6f /etc/glance/policy.json

I am running only Glance v2 API.

In this demo, as an un-privileged user, I will list all glance images, from all tenants, and they are all marked 'private'.

(as admin):
[root@vm013 ~]# openstack role assignment list --effective --names |grep jonathan
| user | jonathan@Default | | ozoneaq@ndc | | False |

(as jonathan):
[root@vm013 ~]# . keystonerc_jonathan
[root@vm013 ~]# printenv |grep OS_ |sort
OS_AUTH_URL=https://keystone.gpcprod:5000/v3
OS_CACERT=/etc/openldap/cacerts/gpcprod_root_ca.pem
OS_IDENTITY_API_VERSION=3
OS_PASSWORD=XXXXXXXXXXXXXXXXXX
OS_PROJECT_DOMAIN_NAME=NDC
OS_PROJECT_NAME=ozoneaq
OS_USER_DOMAIN_NAME=Default
OS_USERNAME=jonathan
OS_VOLUME_API_VERSION=3

[root@vm013 ~]# openstack image list
+--------------------------------------+-----------------------------------+--------+
| ID | Name | Status |
+--------------------------------------+-----------------------------------+--------+
| 0099a343-1376-49f4-85f9-795624fb2ce8 | CentOS-7-x86_64-GenericCloud-1808 | active |
| 53d7c007-318b-4dad-b7cb-38b1dd31f884 | Ubuntu1604-180919 | active |
| 482f52ca-e56c-4555-a0e3-93eb491db389 | Ubuntu1604-20181016 | active |
| 212aaf3c-18f6-4327-8a11-c726c2e21780 | Ubuntu1804-20181016 | active |
| 051d2fff-6b90-4321-9c64-c613f0ddf3da | Windows2016Std-20181003r4 | active |
| ac6baa7c-fd2f-48e2-84e0-37a86f623e38 | Windows2016std-20181003r2 | active |
| 2264c6b9-40e7-492d-a5bc-dd11a7b4ee10 | Windows2016std-20181004 | active |
| 6d865748-ae7a-4c43-9d01-bc35c9002fd9 | Windows2016std-20181004r2 | active |
| 26ba1766-aa67-4b1b-81cd-90dda8d41384 | WindowsServer2016-20180926 | active |
| 3fc3c155-c7a2-4556-a5d0-de7eff208d7d | WindowsStd2016-20181010 | active |
| b6d161ca-e03b-46c5-95a0-5fe31723c5c7 | centos7-201810100 | active |
| 8bdc33be-1eb5-429b-b0ca-682b24df45f0 | centos7-gi-build-test1 | active |
| 34a915b8-cca6-45c3-9348-5e15dace444f | cirros | active |
| 84102d5c-1641-47bb-b727-a59e707e871c | keyshotslave-1604-snap2 | active |
| cedf9ae7-6adc-44d4-b7cb-d5664ea3fef0 | keyshotslave1604-snap1 | active |
| be4dbd67-d56f-41dd-8378-8aa6ca064f55 | mm-cirros-test | active |
| be67cf99-b545-4a91-a3d8-fe9f26a8854d | mm-cirros-test2 | active |
| a8dfd028-5911-4178-a77d-bb3da8996372 | mm-test-image4 | active |
| b6d9d44d-2e3c-48a9-9bf5-b6fca20979f9 | testt2-snap | active |
| 1c401eea-0e6e-475b-9a46-ffbfb388ca35 | ubuntu1804-180919 | active |
+--------------------------------------+-----------------------------------+--------+
[root@vm013 ~]# openstack image show cirros
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checksum | 443b7623e27ecf03dc9e01ee93f67afe |
| container_format | bare |
| created_at | 2018-09-17T13:43:13Z |
| disk_format | raw |
| file | /v2/images/34a915b8-cca6-45c3-9348-5e15dace444f/file |
| id | 34a915b8-cca6-45c3-9348-5e15dace444f |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros |
| owner | 6e6d8ff081014c679f18ad4b818ffd4c |
| properties | direct_url='file:///var/lib/glance/images/34a915b8-cca6-45c3-9348-5e15dace444f', locations='[{u'url': u'file:///var/lib/glance/images/34a915b8-cca6-45c3-9348-5e15dace444f', u'metadata': {u'mountpoint': u'/var/lib/glance/images', u'type': u'nfs', u'id': u'gpc-b32-na-01', u'share_location': u'nfs://gpc-b32-na-01/glance'}}]' |
| protected | False |
| schema | /v2/schemas/image |
| size | 12716032 |
| status | active |
| tags | |
| updated_at | 2018-09-17T13:49:18Z |
| virtual_size | None |
| visibility | private |
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

So you can see that my un-privileged user jonathan (role:user) just displayed the private image 'cirros' from tenant 6e6d8ff081014c679f18ad4b818ffd4c. User 'jonathan' is not a member of that tenant.

(as admin):
[root@vm013 ~]# openstack project list |grep 6e6d8ff081014c679f18ad4b818ffd4c
| 6e6d8ff081014c679f18ad4b818ffd4c | gpcadm |

Perhaps even stranger, as my admin user (role:admin, in admin tenant), I cannot set the visibility of an image to 'public':

[root@vm013 ~]# openstack image set --public cirros
403 Forbidden: You are not authorized to complete publicize_image action. (HTTP 403)

My /etc/glance/policy.json is identical to the reference one, here: https://raw.githubusercontent.com/openstack/glance/master/etc/policy.json

Jeremy Stanley (fungi) wrote :

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

information type: Public → Public Security
Changed in ossa:
status: New → Incomplete
Jonathan Mills (jonmills-t) wrote :
Download full text (4.5 KiB)

This issue was posted to openstack-operators, where some responders said they were unable to duplicate the problem, and that's a big concern of mine. If it is a misconfiguration of my cluster, I still want to understand what went wrong and how to fix it, because this is a pretty serious problem. To shed a little light on our installation procedure, what we do is we provision our bare metal using xCAT, after which we apply RPM packages from the CentOS Cloud repo (http://mirror.centos.org/centos/7/cloud/x86_64/openstack-queens/) and configure services strictly according to the openstack docs (e.g. https://docs.openstack.org/glance/queens/install/install-rdo.html). We tweak the config files to use HTTPS and speak to our endpoints, etc, but there is really nothing exotic about our setup. We were running glance under uwsgi, behind apache+mod_proxy, but as part of our troubleshooting we reverted to using the openstack-glance-api systemd unit file that launches the old python2 eventlet -- there was no change in the exhibited behavior though.

This is our /etc/glance/glance-api.conf:

[DEFAULT]
show_image_direct_url = true
show_multiple_locations = true
location_strategy = store_type
enable_v1_api = false
enable_v2_api = true
enable_v1_registry = false
image_cache_dir = /var/lib/glance/images/image-cache
node_staging_uri = file:///var/lib/glance/images/tmp
use_user_token = true
auth_url = https://keystone.gpcprod:5000/v3
bind_host = 10.2.3.123
bind_port = 9292
workers = 6
http_keepalive = true
client_socket_timeout = 900
backlog = 4096
cert_file = /etc/glance/ssl/glance.pem
key_file = /etc/glance/ssl/glance.key
debug = true
log_file = /var/log/glance/api.log
log_dir = /var/log/glance
use_syslog = False
publish_errors = true
syslog_log_facility = LOG_USER
transport_url = rabbit://user@<email address hidden>:5671,user:<email address hidden>:5671,user:<email address hidden>:5671/

[cors]

[database]
connection = mysql+pymysql://glance:<email address hidden>/glance
connection_recycle_time=200

[glance_store]
stores = file,http
default_store = file
https_ca_certificates_file = /etc/openldap/cacerts/gpcprod_root_ca.pem
https_insecure = false
cinder_catalog_info = volumev3:cinderv3:publicURL
cinder_os_region_name = RegionOne
cinder_ca_certificates_file = /etc/openldap/cacerts/gpcprod_root_ca.pem
cinder_api_insecure = false
filesystem_store_metadata_file = /etc/glance/metadata.json
filesystem_store_datadir = /var/lib/glance/images
filesystem_store_file_perm = 0644
os_region_name=RegionOne

[image_format]
container_formats = ami,ari,aki,bare,ovf,ova
disk_formats = ami,ari,aki,vhd,vmdk,raw,qcow2,vdi,iso

[keystone_authtoken]
auth_uri = https://keystone.gpcprod:5000/v3
auth_url = https://keystone.gpcprod:5000/v3
cafile = /etc/openldap/cacerts/gpcprod_root_ca.pem
insecure = false
region_name = RegionOne
memcached_servers = 10.2.3.117:11211,10.2.3.118:11211,10.2.3.119:11211
token_cache_time = 1800
revocation_cache_time = 1800
memcache_security_strategy = None
memcache_pool_dead_retry = 300
memcache_pool_socket_timeout = 3
auth_type=password
username = glance
password = XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
project_domain_name = default
user...

Read more...

Jonathan Mills (jonmills-t) wrote :

Okay, we believe we understand the problem now.

According to the documentation (https://docs.openstack.org/glance/queens/configuration/glance_api.html), 'keystone' is the Default flavor of the [paste_deploy] section. In most other cases I've seen, when something is a default, it is safe to leave an INI section blank.

In our Queens (and Pike) configurations, we had a blank [paste_deploy] INI section. But we noticed that, in our older Mitaka rack, [paste_deploy] contained 'flavor = keystone'. After adding that value to [paste_deploy] in Queens, immediately image visibility works as expected.

Nevertheless, we still consider this a bug, either in the code or in the documentation. The docs must either insist that the use fill this in (or likely break the anticipated, normal behavior of Glance, and probably NOT say that Keystone is default); or the code needs to change to really use keystone as default in glance-api-paste.ini independent of any INI block or user intervention.

Thanks,

Jonathan

Michael Moore (mdmoore2) wrote :
Jeremy Stanley (fungi) wrote :

Since this bug is already public, related patches can be pushed directly to review.openstack.org (and it's a trivial enough patch that if someone wants to do that for Michael I don't think we need to worry about ICLA coverage for it, but make sure to git-am it so his authorship is correctly preserved).

Michael Moore (mdmoore2) wrote :

@ Jeremey I have just figured out how to deal with gerrit and believe it's submitted.

https://review.openstack.org/613671

Changed in glance:
assignee: nobody → Michael Moore (mdmoore2)
status: New → Fix Committed
Brian Rosmaita (brian-rosmaita) wrote :

Change is still under review.

Changed in glance:
importance: Undecided → Medium
milestone: none → stein-2
status: Fix Committed → In Progress
Brian Rosmaita (brian-rosmaita) wrote :

Adjusted the bug title to reduce the freakout effect.

summary: - non-admin users can see all tenants' images even when image is private
+ default paste_deploy.flavor is none, but config file text implies it is
+ 'keystone' (was: non-admin users can see all tenants' images even when
+ image is private)

The problem is that the way a "sample_default" value is displayed (as opposed to an actual "default" value) is misleading. This has been fixed in oslo.config master (fix not yet released):
https://review.openstack.org/#/c/607353/

We use sample_default for several config values, so regardless of what's decided about changing the default value for paste_deploy.flavor, part of the fix for this bug should be a patch to update requirements.txt and lower-constraints.txt in glance once oslo.config 6.6.3 is released and upper-constraints.txt has been updated in the requirements repo.

I was having serious deja vu yesterday when looking at this. This is a duplicate of https://bugs.launchpad.net/glance/+bug/1711117 . Will mark the old one as a duplicate of this (which is a bit weird, but looks like the discussion on this bug explains that it's actually sort of fixed now).

Change abandoned by Michael Moore (<email address hidden>) on branch: master
Review: https://review.opendev.org/613671
Reason: as mentioned by Brian, resolved in https://review.openstack.org/#/c/607353/

To post a comment you must log in.
This report contains Public Security information  Edit
Everyone can see this security related information.

Duplicates of this bug

Other bug subscribers