passlib 1.7.0 deprecates sha512_crypt.encrypt()

[Mike Bayer]

tests are failing due to a new deprecation warning:

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "keystone/tests/unit/test_backend_sql.py", line 59, in setUp
        self.load_fixtures(default_fixtures)
      File "keystone/tests/unit/core.py", line 754, in load_fixtures
        user_copy = self.identity_api.create_user(user_copy)
      File "keystone/common/manager.py", line 123, in wrapped
        __ret_val = __f(*args, **kwargs)
      File "keystone/identity/core.py", line 410, in wrapper
        return f(self, *args, **kwargs)
      File "keystone/identity/core.py", line 420, in wrapper
        return f(self, *args, **kwargs)
      File "keystone/identity/core.py", line 925, in create_user
        ref = driver.create_user(user['id'], user)
      File "keystone/common/sql/core.py", line 429, in wrapper
        return method(*args, **kwargs)
      File "keystone/identity/backends/sql.py", line 121, in create_user
        user = utils.hash_user_password(user)
      File "keystone/common/utils.py", line 129, in hash_user_password
        return dict(user, password=hash_password(password))
      File "keystone/common/utils.py", line 136, in hash_password
        password_utf8, rounds=CONF.crypt_strength)
      File "/var/lib/jenkins/workspace/openstack_gerrit/keystone/.tox/sqla_py27/lib/python2.7/site-packages/passlib/utils/decor.py", line 190, in wrapper
        warn(msg % tmp, DeprecationWarning, stacklevel=2)
    DeprecationWarning: the method passlib.handlers.sha2_crypt.sha512_crypt.encrypt() is deprecated as of Passlib 1.7, and will be removed in Passlib 2.0, use .hash() instead.

[Lance Bragstad (lbragstad)]

Looks like this is going to require a version bump of passlib in order to fix. Right now we require anything greater than 1.6 [0], but if we switch to using `passlib.hash.sha512_crypt.using().hash()` we will have to bump our version requirement to 1.7, I think.

[0] https://github.com/openstack/keystone/blob/6a67937b8703b61c4084eb4c8dd908e3cea20c35/requirements.txt#L20

[Lance Bragstad (lbragstad)]

Yep - I've confirmed that we will need to bump our version requirements.

http://cdn.pasteraw.com/sgub1owivwivycdbvu4925d6qwkzbr9

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/401328

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/403514

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/401328
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=1df211ad78790685839a711559ded1af8f492ce9
Submitter: Jenkins
Branch: master

commit 1df211ad78790685839a711559ded1af8f492ce9
Author: Lance Bragstad <email address hidden>
Date: Wed Nov 23 15:25:06 2016 +0000

    ignore deprecation warning for .encrypt()

    A new release of passlib 1.7.0 has deprecated the encrypt() method in
    favor of using hash() instead. This commit updates our usage to use
    the recommended way by passlib.

    Change-Id: I69262f2793bfa0c284868a3e6d4aba71ad622ca0
    Partial-Bug: 1644263

[Mike Bayer (zzzeek)]

re-running my gerrit, I am now seeing these errors:

==============================
Failed 4 tests - output below:
==============================

keystone.tests.unit.test_backend_sql.SqlIdentity.test_authenticate_and_get_roles_no_metadata
--------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "keystone/tests/unit/identity/test_backends.py", line 86, in test_authenticate_and_get_roles_no_metadata
        password=user['password'])
      File "keystone/common/manager.py", line 123, in wrapped
        __ret_val = __f(*args, **kwargs)
      File "keystone/notifications.py", line 487, in wrapper
        result = f(wrapped_self, request, user_id, *args, **kwargs)
      File "keystone/identity/core.py", line 410, in wrapper
        return f(self, *args, **kwargs)
      File "keystone/identity/core.py", line 420, in wrapper
        return f(self, *args, **kwargs)
      File "keystone/identity/core.py", line 880, in authenticate
        ref = driver.authenticate(entity_id, password)
      File "keystone/identity/backends/sql.py", line 65, in authenticate
        raise AssertionError(_('Invalid user / password'))
    AssertionError: Invalid user / password

Captured pythonlogging:
~~~~~~~~~~~~~~~~~~~~~~~
    Adding cache-proxy 'oslo_cache.testing.CacheIsolatingProxy' to backend.
    Adding cache-proxy 'oslo_cache.testing.CacheIsolatingProxy' to backend.
    Adding cache-proxy 'oslo_cache.testing.CacheIsolatingProxy' to backend.
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x7ff17a497d10> acquired
    Calling creation function
    Released creation lock

keystone.tests.unit.test_backend_sql.SqlIdentity.test_authenticate
------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "keystone/tests/unit/identity/test_backends.py", line 65, in test_authenticate
        password=self.user_sna['password'])
      File "keystone/common/manager.py", line 123, in wrapped
        __ret_val = __f(*args, **kwargs)
      File "keystone/notifications.py", line 487, in wrapper
        result = f(wrapped_self, request, user_id, *args, **kwargs)
      File "keystone/identity/core.py", line 410, in wrapper
        return f(self, *args, **kwargs)
      File "keystone/identity/core.py", line 420, in wrapper
        return f(self, *args, **kwargs)
      File "keystone/identity/core.py", line 880, in authenticate
        ref = driver.authenticate(entity_id, password)
      File "keystone/identity/backends/sql.py", line 65, in authenticate
        raise AssertionError(_('Invalid user / password'))
    AssertionError: Invalid user / password

Captured pythonlogging:
~~~~~~~~~~~~~~~~~~~~~~~
    Adding cache-proxy 'oslo_cache.testing.CacheIsolatingProxy' to backend.
    Adding cache-proxy 'oslo_cache.testing.CacheIsolatingProxy' to backend.
    Adding cache-proxy 'oslo_cache.testing.CacheIsolatingProxy' to backend.
    NeedRegenerationException
    no ...

[Mike Bayer (zzzeek)]

the gate tests for the above patch seem to have been run against passlib==1.6.5. anyone else testing against 1.7?

[Steve Martinelli (stevemar)]

Mike, https://review.openstack.org/#/c/403514/ should be using passlib 1.7 (via the requirements dependency)

[Mike Bayer (zzzeek)]

ah, OK that's not merged yet, carry on

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/403815

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/403815
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=7e9039bf516268a6ecde36e2f46d150ba0415c69
Submitter: Jenkins
Branch: master

commit 7e9039bf516268a6ecde36e2f46d150ba0415c69
Author: Lance Bragstad <email address hidden>
Date: Mon Nov 28 17:15:21 2016 +0000

    Make try/except work for passlib 1.6 and 1.7

    Change-Id: I126410ce6a57dd298ea5b62a035174eb35d2c138
    Related-Bug: 1644263

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/403514
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=71cde670d5b7f2e9e16d860545d0c36aee115dad
Submitter: Jenkins
Branch: master

commit 71cde670d5b7f2e9e16d860545d0c36aee115dad
Author: Steve Martinelli <email address hidden>
Date: Mon Nov 28 01:22:08 2016 -0500

    Use sha512.hash() instead of .encrypt()

    Now that we have switched to passlib 1.7.0, remove the temporary
    workaround and use the new function.

    Change-Id: Id574221f65d72a763b8205df0891b6e300856230
    Depends-On: I6525dc8cf305ae03b81a53ac7fd06bf63d4a6d48
    Closes-Bug: 1644263

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 11.0.0.0b2 development milestone.