auth tokens logged at INFO level

[Matthew Edmonds]

This issue is being treated as a potential security risk under embargo. Please do not make any public mention of embargoed (private) security vulnerabilities before their coordinated publication by the OpenStack Vulnerability Management Team in the form of an official OpenStack Security Advisory. This includes discussion of the bug or associated fixes in public forums such as mailing lists, code review systems and bug trackers. Please also avoid private disclosure to other individuals not already approved for access to this information, and provide this same reminder to those who are made aware of the issue prior to publication. All discussion should remain confined to this private bug report, and any proposed fixes should be added to the bug as attachments.

--

swiftclient is logging X-Auth-Token in clear text, as seen here:

2016-02-08 14:22:19.905 INFO swiftclient [-] REQ: curl -i http://9.47.82.114:8080/v1/AUTH_cf4d84971239483a89aada19ecbcdba8/powervm_nvram/87627eaf-ee78-44e7-9263-14d256933dd2 -I -H "X-Auth-Token: f1bb0c41234942f1aabd96218ad1c372"

This is similar to https://bugs.launchpad.net/python-swiftclient/+bug/1470740 , but worse because that earlier bug report was only at DEBUG level. This is INFO.

[Tristan Cacqueray (tristan-cacqueray)]

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

[Alistair Coles (alistair-coles)]

I can reproduce this with a swift command that results ina *failed* request - the failed request curl command is logged at the higher INFO level, other curl commands are logged at DEBUG level:

Successful requests:

swift@anc-vm-10:~/swift$ swift --debug stat 2>&1 |grep curl
DEBUG:swiftclient:REQ: curl -i http://localhost:8080/auth/v1.0 -X GET
DEBUG:swiftclient:REQ: curl -i http://localhost:8080/v1/AUTH_test -I -H "X-Auth-Token: AUTH_tkb6e4b1965cd54d1ba42f3626889841d6"

Failed request:

swift@anc-vm-10:~/swift$ swift --debug stat bogus_container 2>&1 |grep curl
DEBUG:swiftclient:REQ: curl -i http://localhost:8080/auth/v1.0 -X GET
INFO:swiftclient:REQ: curl -i http://localhost:8080/v1/AUTH_test/bogus_container -I -H "X-Auth-Token: AUTH_tkb6e4b1965cd54d1ba42f3626889841d6"

[Matthew Edmonds (edmondsw)]

I don't really think this is a duplicate because the linked bug is complaining about these messages in DEBUG logs, and this is complaining about INFO logging. It is much worse to be logging sensitive information in INFO logging, which is on by default. Though the fix will probably be the same.

[Tristan Cacqueray (tristan-cacqueray)]

Password leak in logs file (minus DEBUG mode) usually warrants an OpenStack Security Advisory. Should I add the OSSA to bug 1516692 ? Otherwise I agree with Matthew, this isn't exactly a duplicate and we could work out the advisory part here.

Anyway, John, do you really think this need to be kept private, or could we remove the privacy setting ?

[clayg (clay-gerrard)]

I don't think the loglevel matters - the tokens shouldn't go out to a root logger at *any* level unless the client in control of the logging has configured it such that a) it knows the output of logging won't go to disk and b) it has explicitly requested that auth tokens not be scrubbed.

I think it doesn't matter *at all* what our command line client logs and what at what level *to the console* - I think it's an annoying and dangerous default configuration for programatic access to emit tokens to the root logger.

But this issue is already widely documented in the duplicated public bug - there's no reason to leave this private, or even to leave it open.

FWIW, the workaround for anyone using the client with programatic access is to make logging for just the python-swiftclient logger more quiet:

logging.getLogger("swiftclient").setLevel(logging.WARNING)

^ not optimal, but far better than logging all the tokens until a proper fix can be developed.

[Matthew Edmonds (edmondsw)]

FWIW I'd agree with you Clay, but in the past the Security Group has argued differently and decided to release OSSAs for non-debug cases and OSSN's for debug cases.

I do agree with making this public, as it seems the information is out there.

[Jeremy Stanley (fungi)]

Honestly, the only reason the VMT makes a distinction between sensitive information disclosed in DEBUG log level and other log levels is that DEBUG is typically a non-default configuration, and also there are (or at least were historically) so many leaks of information in DEBUG level logs across most OpenStack projects that finding them was like shooting fish in a barrel (also some projects, like Horizon, had extremely visible warnings about that fact in their documentation). If this situation has changed for the better, I'm happy to revisit that choice.

[Matthew Edmonds (edmondsw)]

Jeremy, I do think it's gotten a lot better, at least in the ones I'm more familiar with (keystone, nova, glance, cinder, neutron, ceilometer). I'd say it's worth revisiting.

[Tristan Cacqueray (tristan-cacqueray)]

I've removed the privacy status based on above comments.

And I'm not against removing the "non-DEBUG" criteria, but this have to be well defined in openstack documentation and projects may have to implement a non default option to remove the password/token hidden from debug if the full trace is required (such as the dovecot auth_debug_passwords option).

[Matthew Edmonds (edmondsw)]

I think this was fixed in swiftclient under bug #1516692. What are we doing about an OSSA to get this off the books?