sha512_crypt is insufficient, use pbkdf2_sha512 for password hashing

[Morgan Fainberg]

Keystone uses sha512_crypt for password hashing. This is insufficient and provides limited protection (even with 10,000 rounds) against brute-forcing of the password hashes (especially with FPGAs and/or GPU processing).

The correct mechanism is to use bcrypt, scrypt, or pbkdf2_sha512 instead of sha512_crypt.

This bug is marked as public security as bug #1543048 has already highlighted this issue.

[Morgan Fainberg (mdrnstm)]

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/438808

[Morgan Fainberg (mdrnstm)]

As an update based upon the comments and discussion in keystone here is the course of action:

* No backports

* Pike will be updated to support pbkfd2_sha512, bcrypt, and scrypt (configurable) - default will be bcrypt

* For rolling upgrade purposes, keystone will still write sha512_crypt passwords to the old column, new column will be created for the new password hashes. This old crypt hash will be disable-able from being written via configuration option.

* IN Q release, keystone will cease to write sha512_crypt and the configuration option will be deprecated for removal/removed (that toggles sha512_crypt writing).

This means OSSA can be closed, OSSN task can be opened if OSSG would like to issue an OSSN for this.

While sha512_crypt and sha256_crypt are used in many cases, these are in places that are typically more secure than web-facing applications (shadow file) where pbkfd2, bcrypt, and scrypt really shine and start providing significantly more protection against off-line brute force especially since databases are more likely to be breached as they are more often accessible from more locations than the shadow/filesystem is).

[OpenStack Infra (hudson-openstack)]

Change abandoned by Morgan Fainberg (<email address hidden>) on branch: master
Review: https://review.openstack.org/438808
Reason: Abandoning, no backports needed can go with a more comprehensive fix

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/438701
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8ad765e0230ceeb5ca7c36ec3ed6d25c57b22c9d
Submitter: Jenkins
Branch: master

commit 8ad765e0230ceeb5ca7c36ec3ed6d25c57b22c9d
Author: Morgan Fainberg <email address hidden>
Date: Mon Feb 27 13:06:07 2017 -0800

    Support new hashing algorithms for securely storing password hashes

    Support bcrypt, pbkdf2_sha512, or scrypt in password hashing for
    passwords managed within keystone. sha512_crypt is insufficient to
    hash passwords in a secure way for storage in the DB. Keystone defaults
    now to using bcrypt but can handle scrypt and pbkdf2_sha512 with a number
    of tuning options if desired.

    Closes-bug: #1543048
    Closes-bug: #1668503
    Change-Id: Id05026720839d94de26d0e44631deb34bcc0e610

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 12.0.0.0b2 development milestone.

[Tristan Cacqueray (tristan-cacqueray)]

Adding OSSN task based on comment #3

[Luke Hinds (lhinds)]

Couple of Q's...

For the OSSN what would the 'recommended actions' be to update to Pike?

Is it a seamless crossover going from sha512_crypt to bcrypt, scrypt, or pbkdf2_sha512, or would passwords need to be regenerated (thinking in this instance of an operator upgrading from a previous release to Pike) ?

[Morgan Fainberg (mdrnstm)]

All new/updated/changes passwords (after upgrade) would be bcrypt hashed,
old passwords remain sha512_crypt. An operator may want to force password
changes.

On Aug 15, 2017 00:11, "Luke Hinds" <email address hidden> wrote:

> Couple of Q's...
>
> For the OSSN what would the 'recommended actions' be to update to Pike?
>
> Is it a seamless crossover going from sha512_crypt to bcrypt, scrypt, or
> pbkdf2_sha512, or would passwords need to be regenerated (thinking in
> this instance of an operator upgrading from a previous release to Pike)
> ?
>
> ** Changed in: ossn
> Assignee: (unassigned) => Luke Hinds (lhinds)
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> Matching subscriptions: Private security bugs
> https://bugs.launchpad.net/bugs/1668503
>
> Title:
> sha512_crypt is insufficient, use pbkdf2_sha512 for password hashing
>
> Status in OpenStack Identity (keystone):
> Fix Released
> Status in OpenStack Identity (keystone) mitaka series:
> Won't Fix
> Status in OpenStack Identity (keystone) newton series:
> Won't Fix
> Status in OpenStack Identity (keystone) ocata series:
> Won't Fix
> Status in OpenStack Identity (keystone) pike series:
> Fix Released
> Status in OpenStack Security Advisory:
> Won't Fix
> Status in OpenStack Security Notes:
> New
>
> Bug description:
> Keystone uses sha512_crypt for password hashing. This is insufficient
> and provides limited protection (even with 10,000 rounds) against
> brute-forcing of the password hashes (especially with FPGAs and/or GPU
> processing).
>
> The correct mechanism is to use bcrypt, scrypt, or pbkdf2_sha512
> instead of sha512_crypt.
>
> This bug is marked as public security as bug #1543048 has already
> highlighted this issue.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/keystone/+bug/1668503/+subscriptions
>

[Morgan Fainberg (mdrnstm)]

** or to whichever hash is configured if not default.

On Aug 15, 2017 07:48, "Morgan Fainberg" <email address hidden> wrote:

> All new/updated/changes passwords (after upgrade) would be bcrypt hashed,
> old passwords remain sha512_crypt. An operator may want to force password
> changes.
>
> On Aug 15, 2017 00:11, "Luke Hinds" <email address hidden> wrote:
>
>> Couple of Q's...
>>
>> For the OSSN what would the 'recommended actions' be to update to Pike?
>>
>> Is it a seamless crossover going from sha512_crypt to bcrypt, scrypt, or
>> pbkdf2_sha512, or would passwords need to be regenerated (thinking in
>> this instance of an operator upgrading from a previous release to Pike)
>> ?
>>
>> ** Changed in: ossn
>> Assignee: (unassigned) => Luke Hinds (lhinds)
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> Matching subscriptions: Private security bugs
>> https://bugs.launchpad.net/bugs/1668503
>>
>> Title:
>> sha512_crypt is insufficient, use pbkdf2_sha512 for password hashing
>>
>> Status in OpenStack Identity (keystone):
>> Fix Released
>> Status in OpenStack Identity (keystone) mitaka series:
>> Won't Fix
>> Status in OpenStack Identity (keystone) newton series:
>> Won't Fix
>> Status in OpenStack Identity (keystone) ocata series:
>> Won't Fix
>> Status in OpenStack Identity (keystone) pike series:
>> Fix Released
>> Status in OpenStack Security Advisory:
>> Won't Fix
>> Status in OpenStack Security Notes:
>> New
>>
>> Bug description:
>> Keystone uses sha512_crypt for password hashing. This is insufficient
>> and provides limited protection (even with 10,000 rounds) against
>> brute-forcing of the password hashes (especially with FPGAs and/or GPU
>> processing).
>>
>> The correct mechanism is to use bcrypt, scrypt, or pbkdf2_sha512
>> instead of sha512_crypt.
>>
>> This bug is marked as public security as bug #1543048 has already
>> highlighted this issue.
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/keystone/+bug/1668503/+subscriptions
>>
>

[Luke Hinds (lhinds)]

ack, thanks Morgan..I will start on the OSSN.

[Jeremy Stanley (fungi)]

It's worth noting that on review of the original source, keystone.common.password_hashing.hash_password() was not using a bare SHA-512 hash like hashlib.sha512() but instead already performed key derivation via passlib.hash.sha512_crypt.hash(), a wrapper for the vaguely-named SHA-512 method for glibc's crypt() function. Per the ML thread starting at http://www.openwall.com/lists/oss-security/2017/09/17/2 the change itself is at least for the better, but the risk from the previous implementation is not nearly as severe as this bug and the subsequent OSSN would seem to suggest.

Apologies if my poorly-researched comment in bug 1543048 led to a fire drill, I'll propose an errata revision to OSSN-0081 this week.

[Morgan Fainberg (mdrnstm)]

I also apologize if I didn't make it clear it was pbkdf still with sha512, just that sha512_crypt (even pbkdf) is not sufficient for applications (e.g. exposing your shadow file on linux *is* still a compromise). The implication is that there is significantly more surface area for Keystone to deal with than a system shadow file, therefore we should be using bcrypt, scrypt, or at *least* pbkdf2 instead of sha512_crypt.

tl;dr I may have also missed on communicating sha512_crypt is pbkdf rather than bare sha512 hashing.