Debug and -vv outputs user password in plain text

Bug #1630822 reported by Kris Lindgren
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Security Advisory
Won't Fix
Undecided
Unassigned
python-openstackclient
Fix Released
Undecided
Matt Riedemann

Bug Description

Running an openstackclient command with --debug or -vv unexpectedly outputs the password in plaintext, multiple times. This is going to cause people to pastebin their user credentials while using openstack client.

The password is logged in plain text in 3 locations:
1.) Under options: (second line of output)
2.) Under defaults: (third line of output)
3.) Under the Using paramters {'username': '<username>'. 'tenant_name': '<tenantname>', 'password': '<password>

I believe that all output from the tool should have sensitive information scrubbed from the output.

Specific output as an example:

$ openstack -vv availability zone list
START with options: ['-vv', 'availability', 'zone', 'list']
options: Namespace(access_token_endpoint='', auth_type='', auth_url='https://openstack-dev.int.godaddy.com:35357/v2.0/', cacert='', client_id='', client_secret='', cloud='', debug=False, default_domain='default', deferred_help=False, domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='', log_file=None, os_compute_api_version='', os_data_processing_api_version='1.1', os_dns_api_version='', os_identity_api_version='', os_image_api_version='', os_network_api_version='', os_object_api_version='', os_project_id=None, os_project_name=None, os_queues_api_version='1.1', os_volume_api_version='', password=mypasswordinplaintext, project_domain_id='', project_domain_name='', project_id='', project_name='openstack', protocol='', region_name='', scope='', service_provider_endpoint='', timing=False, token='', trust_id='', url='', user_domain_id='', user_domain_name='', user_id='', username='myusername', verbose_level=3, verify=None)
defaults: {'auth_type': 'password', 'compute_api_version': '2', 'database_api_version': '1.0', 'api_timeout': None, 'baremetal_api_version': '1', 'interface': None, 'image_api_use_tasks': False, 'endpoint_type': 'public', 'floating_ip_source': 'neutron', 'key': None, 'cacert': None, 'network_api_version': '2', 'object_api_version': '1', 'image_api_version': '1', 'verify': True, 'identity_api_version': '2', 'volume_api_version': '1', 'cert': None, 'secgroup_source': 'neutron', 'disable_vendor_agent': {}}
cloud cfg: {'auth_type': 'password', 'compute_api_version': '2', 'database_api_version': '1.0', 'data_processing_api_version': '1.1', 'network_api_version': '2', 'object_api_version': '1', 'queues_api_version': '1.1', 'verify': True, 'timing': False, 'verbose_level': 3, 'region_name': '', 'api_timeout': None, 'baremetal_api_version': '1', 'image_api_version': '1', 'auth': {'username': 'myusername', 'tenant_name': 'openstack', 'project_name': 'openstack', 'password': mypasswordinplaintext, 'auth_url': 'https://openstack-dev.int.godaddy.com:35357/v2.0/'}, 'default_domain': 'default', 'image_api_use_tasks': False, 'endpoint_type': 'public', 'floating_ip_source': 'neutron', 'key': None, 'interface': None, 'cacert': None, 'deferred_help': False, 'identity_api_version': '2', 'volume_api_version': '1', 'cert': None, 'secgroup_source': 'neutron', 'debug': False, 'disable_vendor_agent': {}}
compute API version 2, cmd group openstack.compute.v2
network API version 2, cmd group openstack.network.v2
image API version 1, cmd group openstack.image.v1
volume API version 1, cmd group openstack.volume.v1
identity API version 2, cmd group openstack.identity.v2
object_store API version 1, cmd group openstack.object_store.v1
dns API version 2, cmd group openstack.dns.v2
data_processing API version 1.1, cmd group openstack.data_processing.v1
messaging API version 1.1, cmd group openstack.messaging.v1
command: availability zone list -> openstackclient.compute.v2.availability_zone.ListAvailabilityZone
Auth plugin password selected
auth_type: password
Using auth plugin: password
Using parameters {'username': 'myusername', 'tenant_name': 'openstack', 'password': mypasswordinplaintext, 'project_name': 'openstack', 'auth_url': 'https://openstack-dev.int.godaddy.com:35357/v2.0/'}
Get auth_ref
REQ: curl -g -i -X GET https://openstack-dev.int.godaddy.com:35357/v2.0/ -H "Accept: application/json" -H "User-Agent: python-openstackclient"
RESP: [200] Vary: X-Auth-Token Content-Type: application/json Content-Length: 357 x-openstack-request-id: req-feeb6ea9-7936-4428-9628-ee15fd6b831a Connection: close
RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "https://openstack-dev.int.godaddy.com:35357/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}}

Making authentication request to https://openstack-dev.int.godaddy.com:35357/v2.0/tokens
take_action(Namespace(columns=[], formatter='table', long=False, max_width=0, noindent=False, quote_mode='nonnumeric'))
Instantiating compute client for VAPI Version Major: 2, Minor: 0
Making authentication request to https://openstack-dev.int.godaddy.com:35357/v2.0/tokens
REQ: curl -g -i -X GET https://openstack-dev.int.godaddy.com:8774/v2/f48e57277a7a484290ba9afdc49a21a9/os-availability-zone/detail -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}20ca58f7998139291b7cb394503f0cf8e820e38b"
RESP: [200] Content-Type: application/json Content-Length: 200 x-compute-request-id: req-869f2d60-6d4d-499d-8deb-6f2c7ae319d4
RESP BODY: {"availabilityZoneInfo": [{"zoneState": {"available": true}, "hosts": null, "zoneName": "glbt1-dev-lab-zone-1"}, {"zoneState": {"available": true}, "hosts": null, "zoneName": "glbt1-dev-lab-zone-2"}]}

+----------------------+-------------+
| Zone Name | Zone Status |
+----------------------+-------------+
| glbt1-dev-lab-zone-1 | available |
| glbt1-dev-lab-zone-2 | available |
+----------------------+-------------+
clean_up ListAvailabilityZone:
END return value: 0

Revision history for this message
Matt Riedemann (mriedem) wrote :

What version are you using?

Revision history for this message
Matt Riedemann (mriedem) wrote :

From IRC, version is 1.7.2.

Revision history for this message
Matt Riedemann (mriedem) wrote :

And 3.2.0 so newton, so it's probably still a bug.

Revision history for this message
Matt Riedemann (mriedem) wrote :

Looks like this was at least in part fixed back in mitaka:

https://review.openstack.org/#/c/233271/

So there must be something else that's leaking passwords.

Revision history for this message
Xav Paice (xavpaice) wrote :
Download full text (8.4 KiB)

My output from 3.2.0:

$ openstack --debug token issue
START with options: [u'--debug', u'token', u'issue']
options: Namespace(access_token='***', access_token_endpoint='', access_token_type='', auth_type='', auth_url='https://api.cloud.catalyst.net.nz:5000/v2.0', authorization_code='', cacert=None, cert='', client_id='', client_secret='***', cloud='', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', insecure=None, interface='', key='', log_file=None, openid_scope='', os_beta_command=False, os_compute_api_version='', os_identity_api_version='2.0', os_image_api_version='', os_network_api_version='', os_object_api_version='', os_orchestration_api_version='1', os_project_id=None, os_project_name=None, os_volume_api_version='', passcode='', password='***', project_domain_id='', project_domain_name='', project_id='', project_name='openstack-dev.catalyst.net.nz', protocol='', redirect_uri='', region_name='nz-por-1', timing=False, token='***', trust_id='', url='', user_domain_id='', user_domain_name='', user_id='', username='xavuser', verbose_level=3, verify=None)
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'https://api.cloud.catalyst.net.nz:5000/v2.0', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', u'status': u'active', 'verbose_level': 3, 'region_name': 'nz-por-1', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'project_name': 'openstack-dev.catalyst.net.nz'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': '1', 'timing': False, 'password': 'THISISAREALPASSWORDRIGHTHERE', 'cacert': None, u'key_manager_api_version': u'v1', 'deferred_help': False, u'identity_api_version': '2.0', u'volume_api_version': u'2', 'username': 'xavuser', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}}
defaults: {u'auth_type': 'password', u'status': u'active', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', u'image_api_version': u'2', u'metering_api_version': u'2', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'cacert': None, u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', u'key_manager_api_version': u'v1', 'verify': True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'dns_api_version': u'2', u'object_store_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}}
cloud cfg: {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', u'orchestration_api...

Read more...

Revision history for this message
Matt Riedemann (mriedem) wrote :
Changed in python-openstackclient:
status: New → Confirmed
Matt Riedemann (mriedem)
tags: added: mitaka-backport-potential newton-backport-potential
Revision history for this message
Matt Riedemann (mriedem) wrote :
Changed in python-openstackclient:
assignee: nobody → Matt Riedemann (mriedem)
status: Confirmed → In Progress
Revision history for this message
Matt Riedemann (mriedem) wrote :

So we have patches up for this, and I've backported https://review.openstack.org/#/c/382698/ to stable/liberty.

I've added OSSA to this since it's probably worth an advisory given you could be providing your cloud credentials in a paste (bug reports) when someone asks you to recreate a bug via openstack CLI with the --debug option and you just blindly send it all over and it's public.

Revision history for this message
Steve Martinelli (stevemar) wrote :

Looks like in the osc transition to osc-lib and keystoneauth caused this issue to regress.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to osc-lib (master)

Reviewed: https://review.openstack.org/382701
Committed: https://git.openstack.org/cgit/openstack/osc-lib/commit/?id=0a82bd7294ad3148df048d73bfa3f04c7ddca4f2
Submitter: Jenkins
Branch: master

commit 0a82bd7294ad3148df048d73bfa3f04c7ddca4f2
Author: Matt Riedemann <email address hidden>
Date: Wed Oct 5 21:17:39 2016 -0400

    Mask passwords in debug logs for auth_config_hook

    The auth config hook can have credentials in it so
    we have to mask the config before logging it. To
    avoid doing the work of masking the password if we
    aren't going to log it, there is a conditional put
    around the actual debug statement.

    Change-Id: I8e626672ec94fc837610216bccb4354dbdedca17
    Closes-Bug: #1630822

Changed in python-openstackclient:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to osc-lib (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/383432

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-openstackclient (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/383434

Matt Riedemann (mriedem)
tags: removed: newton-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-openstackclient (master)

Reviewed: https://review.openstack.org/382699
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=cd1a412408f068aeef97c1ee368400307fce7733
Submitter: Jenkins
Branch: master

commit cd1a412408f068aeef97c1ee368400307fce7733
Author: Matt Riedemann <email address hidden>
Date: Wed Oct 5 21:11:16 2016 -0400

    Mask passwords in debug logs for auth_config_hook

    The auth config hook can have credentials in it so
    we have to mask the config before logging it. To
    avoid doing the work of masking the password if we
    aren't going to log it, there is a conditional put
    around the actual debug statement.

    Change-Id: I8e626672ec94fc837610216bccb4354dbdedca17
    Closes-Bug: #1630822

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to osc-lib (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/383755

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to osc-lib (master)

Reviewed: https://review.openstack.org/383755
Committed: https://git.openstack.org/cgit/openstack/osc-lib/commit/?id=455eb362a3ec241cfafcbba9d0ee147a493b07f6
Submitter: Jenkins
Branch: master

commit 455eb362a3ec241cfafcbba9d0ee147a493b07f6
Author: Matt Riedemann <email address hidden>
Date: Fri Oct 7 10:06:54 2016 -0400

    Add release note for security bug 1630822

    This should have been included with the original
    fix, but we should have this in the same release
    at least.

    Change-Id: I9ab1f06282ec33034e7b6b11863ea9f9234d6fe0
    Related-Bug: #1630822

Revision history for this message
Tristan Cacqueray (tristan-cacqueray) wrote :

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

Though note that the python-openstackclient doesn't have the vulnerability:managed tag.

Changed in ossa:
status: New → Incomplete
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-openstackclient 3.3.0

This issue was fixed in the openstack/python-openstackclient 3.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/osc-lib 1.2.0

This issue was fixed in the openstack/osc-lib 1.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to osc-lib (stable/newton)

Reviewed: https://review.openstack.org/383432
Committed: https://git.openstack.org/cgit/openstack/osc-lib/commit/?id=422009afdfea90e9351632514adda8dbb1218607
Submitter: Jenkins
Branch: stable/newton

commit 422009afdfea90e9351632514adda8dbb1218607
Author: Matt Riedemann <email address hidden>
Date: Wed Oct 5 21:17:39 2016 -0400

    Mask passwords in debug logs for auth_config_hook

    The auth config hook can have credentials in it so
    we have to mask the config before logging it. To
    avoid doing the work of masking the password if we
    aren't going to log it, there is a conditional put
    around the actual debug statement.

    Conflicts:
            osc_lib/cli/client_config.py

    NOTE(mriedem): The conflict was due to the imports.

    Change-Id: I8e626672ec94fc837610216bccb4354dbdedca17
    Closes-Bug: #1630822
    (cherry picked from commit 0a82bd7294ad3148df048d73bfa3f04c7ddca4f2)

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/osc-lib 1.2.0

This issue was fixed in the openstack/osc-lib 1.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-openstackclient 3.3.0

This issue was fixed in the openstack/python-openstackclient 3.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-openstackclient (stable/newton)

Reviewed: https://review.openstack.org/383434
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=a37addb7b034105ce03ee35a0a1610d191c4b269
Submitter: Jenkins
Branch: stable/newton

commit a37addb7b034105ce03ee35a0a1610d191c4b269
Author: Matt Riedemann <email address hidden>
Date: Wed Oct 5 21:11:16 2016 -0400

    Mask passwords in debug logs for auth_config_hook

    The auth config hook can have credentials in it so
    we have to mask the config before logging it. To
    avoid doing the work of masking the password if we
    aren't going to log it, there is a conditional put
    around the actual debug statement.

    Conflicts:
            openstackclient/common/client_config.py

    NOTE(mriedem): The conflict was due to imports.

    Change-Id: I8e626672ec94fc837610216bccb4354dbdedca17
    Closes-Bug: #1630822
    (cherry picked from commit cd1a412408f068aeef97c1ee368400307fce7733)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-openstackclient 3.2.1

This issue was fixed in the openstack/python-openstackclient 3.2.1 release.

Revision history for this message
Tristan Cacqueray (tristan-cacqueray) wrote :

While it isn't an explicit debug, it seems like "-vv" does enable debug mode. Thus I propose treating this as a class B3 report ( https://security.openstack.org/vmt-process.html#incident-report-taxonomy ).

Jeremy Stanley (fungi)
Changed in ossa:
status: Incomplete → Won't Fix
tags: added: security
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.