py3: inconsistent encoding of token fields

Bug #1832265 reported by Drew Freiberger on 2019-06-10
24
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Undecided
James Page
OpenStack Keystone LDAP integration
High
James Page
Ubuntu Cloud Archive
Status tracked in Stein
Rocky
High
Unassigned
Stein
High
Unassigned
Train
High
Unassigned
keystone (Ubuntu)
High
James Page
Cosmic
High
Unassigned
Disco
High
Unassigned

Bug Description

When using an LDAP domain user on a bionic-rocky cloud within horizon, we are unable to see the projects listed in the project selection drop-down, and are unable to query resources from any projects to which we are assigned the role Member.

It appears that the following log entries in keystone may be helpful to troubleshooting this issue:

(keystone.middleware.auth): 2019-06-10 19:47:02,700 DEBUG RBAC: auth_context: {'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_id': None, 'domain_name': None, 'group_ids': [], 'token': <TokenModel (audit_id=8_4AHHWtSQ2JjTiwU7Kh0g, audit_chain_id=['8_4AHHWtSQ2JjTiwU7Kh0g']) at 0x7fed2c7909b0>, 'user_id': b'd4fb94cfa3ce0f7829d76fe44697488e7765d88e29f5a896f57d43caadb0fad4', 'user_domain_id': '997b3e91271140feb1635eefba7c65a1', 'system_scope': None, 'project_id': None, 'project_domain_id': None, 'roles': [], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []}
(keystone.server.flask.application): 2019-06-10 19:47:02,700 DEBUG Dispatching request to legacy mapper: /v3/users
(keystone.server.flask.application): 2019-06-10 19:47:02,700 DEBUG SCRIPT_NAME: `/v3`, PATH_INFO: `/users/d4fb94cfa3ce0f7829d76fe44697488e7765d88e29f5a896f57d43caadb0fad4/projects`
(routes.middleware): 2019-06-10 19:47:02,700 DEBUG Matched GET /users/d4fb94cfa3ce0f7829d76fe44697488e7765d88e29f5a896f57d43caadb0fad4/projects
(routes.middleware): 2019-06-10 19:47:02,700 DEBUG Route path: '/users/{user_id}/projects', defaults: {'action': 'list_user_projects', 'controller': <keystone.assignment.controllers.ProjectAssignmentV3 object at 0x7fed2ec52ef0>}
(routes.middleware): 2019-06-10 19:47:02,700 DEBUG Match dict: {'user_id': 'd4fb94cfa3ce0f7829d76fe44697488e7765d88e29f5a896f57d43caadb0fad4', 'action': 'list_user_projects', 'controller': <keystone.assignment.controllers.ProjectAssignmentV3 object at 0x7fed2ec52ef0>}
(keystone.common.wsgi): 2019-06-10 19:47:02,700 INFO GET https://keystone.mysite:5000/v3/users/d4fb94cfa3ce0f7829d76fe44697488e7765d88e29f5a896f57d43caadb0fad4/projects
(keystone.common.controller): 2019-06-10 19:47:02,700 DEBUG RBAC: Adding query filter params ()
(keystone.common.authorization): 2019-06-10 19:47:02,700 DEBUG RBAC: Authorizing identity:list_user_projects(user_id=d4fb94cfa3ce0f7829d76fe44697488e7765d88e29f5a896f57d43caadb0fad4)
(keystone.policy.backends.rules): 2019-06-10 19:47:02,701 DEBUG enforce identity:list_user_projects: {'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_id': None, 'domain_name': None, 'group_ids': [], 'token': <TokenModel (audit_id=8_4AHHWtSQ2JjTiwU7Kh0g, audit_chain_id=['8_4AHHWtSQ2JjTiwU7Kh0g']) at 0x7fed2c7909b0>, 'user_id': b'd4fb94cfa3ce0f7829d76fe44697488e7765d88e29f5a896f57d43caadb0fad4', 'user_domain_id': '997b3e91271140feb1635eefba7c65a1', 'system_scope': None, 'project_id': None, 'project_domain_id': None, 'roles': [], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []}
(keystone.common.wsgi): 2019-06-10 19:47:02,702 WARNING You are not authorized to perform the requested action: identity:list_user_projects.

It actually appears elsewhere in the keystone.log that there is a string which has encapsulated bytecode data in it (or vice versa).

(keystone.common.wsgi): 2019-06-10 19:46:59,019 INFO POST https://keystone.mysite:5000/v3/auth/tokens
(sqlalchemy.orm.path_registry): 2019-06-10 19:46:59,021 DEBUG set 'memoized_setups' on path 'EntityRegistry((<Mapper at 0x7fed2eccfc50; RevocationEvent>,))' to '{}'
(sqlalchemy.pool.QueuePool): 2019-06-10 19:46:59,021 DEBUG Connection <pymysql.connections.Connection object at 0x7fed2c7d8320> checked out from pool
(sqlalchemy.pool.QueuePool): 2019-06-10 19:46:59,024 DEBUG Connection <pymysql.connections.Connection object at 0x7fed2c7d8320> being returned to pool
(sqlalchemy.pool.QueuePool): 2019-06-10 19:46:59,024 DEBUG Connection <pymysql.connections.Connection object at 0x7fed2c7d8320> rollback-on-return, via agent
(keystone.auth.core): 2019-06-10 19:46:59,025 DEBUG MFA Rules not processed for user `b'd4fb94cfa3ce0f7829d76fe44697488e7765d88e29f5a896f57d43caadb0fad4'`. Rule list: `[]` (Enabled: `True`).
(keystone.common.wsgi): 2019-06-10 19:46:59,025 ERROR a bytes-like object is required, not 'str'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/keystone/common/wsgi.py", line 148, in __call__
    result = method(req, **params)
  File "/usr/lib/python3/dist-packages/keystone/auth/controllers.py", line 102, in authenticate_for_token
    app_cred_id=app_cred_id, parent_audit_id=token_audit_id)
  File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 116, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/keystone/token/provider.py", line 251, in issue_token
    token_id, issued_at = self.driver.generate_id_and_issued_at(token)
  File "/usr/lib/python3/dist-packages/keystone/token/providers/fernet/core.py", line 61, in generate_id_and_issued_at
    app_cred_id=token.application_credential_id
  File "/usr/lib/python3/dist-packages/keystone/token/token_formatters.py", line 159, in create_token
    protocol_id, access_token_id, app_cred_id
  File "/usr/lib/python3/dist-packages/keystone/token/token_formatters.py", line 483, in assemble
    b_user_id = cls.attempt_convert_uuid_hex_to_bytes(user_id)
  File "/usr/lib/python3/dist-packages/keystone/token/token_formatters.py", line 337, in attempt_convert_uuid_hex_to_bytes
    return (True, cls.convert_uuid_hex_to_bytes(value))
  File "/usr/lib/python3/dist-packages/keystone/token/token_formatters.py", line 290, in convert_uuid_hex_to_bytes
    uuid_obj = uuid.UUID(uuid_string)
  File "/usr/lib/python3.6/uuid.py", line 137, in __init__
    hex = hex.replace('urn:', '').replace('uuid:', '')
TypeError: a bytes-like object is required, not 'str'
(dogpile.lock): 2019-06-10 19:46:59,087 DEBUG value creation lock <dogpile.cache.region.CacheRegion._LockWrapper object at 0x7fed27e994e0> acquired
(dogpile.lock): 2019-06-10 19:46:59,087 DEBUG Calling creation function

Drew Freiberger (afreiberger) wrote :

Subscribed field-high as this is affecting usability of a production cloud.

Drew Freiberger (afreiberger) wrote :

Also of note, we have not seen success in listing LDAP group members, either. We believe all the ldap flags are properly configured for that, but it's also an anomaly. Not sure if it's same bug or different bug.

James Page (james-page) wrote :

The stack trace would indicate that the user_id parameter is binary encoded under py3.

James Page (james-page) wrote :

WARNING You are not authorized to perform the requested action: identity:list_user_projects is probably also pertinent here.

James Page (james-page) wrote :

>>> uuid.UUID(b'bf989240-79be-4155-8473-f12f1d13ed3d')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.7/uuid.py", line 157, in __init__
    hex = hex.replace('urn:', '').replace('uuid:', '')
TypeError: a bytes-like object is required, not 'str'

James Page (james-page) wrote :

user_id is passed right from the top of the stack as part of the authentication context

James Page (james-page) wrote :

Value: b'd4fb94cfa3ce0f7829d76fe44697488e7765d88e29f5a896f57d43caadb0fad4'

James Page (james-page) wrote :

I think this is a bit of a behaviour difference between py2 and py3 - Python 2 will throw a ValueError when passed a binary encoded string, Python 3 throws a TypeError.

James Page (james-page) wrote :

That said I don't think the user_id string should be byte encoded.

tags: added: py3
Drew Freiberger (afreiberger) wrote :

I performed a quick update to add in an exception for TypeError along with the ValueError exception handler, and was able to clean up the traceback, but it still doesn't function as expected.

/usr/lib/python3/dist-packages/keystone/token/token_formatters.py:

    def attempt_convert_uuid_hex_to_bytes(cls, value):
        """Attempt to convert value to bytes or return value.

        :param value: value to attempt to convert to bytes
        :returns: tuple containing boolean indicating whether user_id was
                  stored as bytes and uuid value as bytes or the original value

        """
        try:
            return (True, cls.convert_uuid_hex_to_bytes(value))
        except ValueError:
            # this might not be a UUID, depending on the situation (i.e.
            # federation)
            return (False, value)
        except TypeError:
            # lp#1832265 - py3 uuid may raise TypeError instead of ValueError
            return (False, value)

As I did, I saw the log lines flow through showing user_id as a string, and then immediately after, as a byte-encoded value in the log.

(keystone.common.controller): 2019-06-11 18:12:07,270 DEBUG RBAC: Adding query filter params ()
(keystone.common.authorization): 2019-06-11 18:12:07,270 DEBUG RBAC: Authorizing identity:list_user_projects(user_id=d4fb94cfa3ce0f7829d76fe44697488e7765d88e29f5a896f57d43caadb0fad4)
**** somewhere in here is where user_id ends up getting bytestring encoded ****
(keystone.policy.backends.rules): 2019-06-11 18:12:07,270 DEBUG enforce identity:list_user_projects: {'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_id': None, 'domain_name': None, 'group_ids': [], 'token': <TokenModel (audit_id=azlKaSkfR3arBR5tGIzRlw, audit_chain_id=['azlKaSkfR3arBR5tGIzRlw']) at 0x7fd2e1d53e80>, 'user_id': b'd4fb94cfa3ce0f7829d76fe44697488e7765d88e29f5a896f57d43caadb0fad4', 'user_domain_id': '997b3e91271140feb1635eefba7c65a1', 'system_scope': None, 'project_id': None, 'project_domain_id': None, 'roles': [], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []}

Drew Freiberger (afreiberger) wrote :

Taking a cue from def disassemble, I added in an exception handler for this binary_type user_id validation in (keystone.identity.backends.ldap.core). This appears to resolve the LDAP access issues within horizon as far as I'm able to tell. Basically, this is handling any call to the ldap UserApi subclass call for self.user.get(*attribute), to check for byte-encoded values coming from the ldap backend and strip them.

/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/core.py diff:

*** a/core.py 2019-06-11 20:29:23.148303839 +0000
--- b/core.py 2019-06-11 20:27:23.854658160 +0000
*************** class UserApi(common_ldap.EnabledEmuMixI
*** 308,313 ****
--- 308,318 ----
      def get(self, user_id, ldap_filter=None):
          obj = super(UserApi, self).get(user_id, ldap_filter=ldap_filter)
          obj['options'] = {} # options always empty
+
+ # lp#1832265
+ if six.PY3 and isinstance(obj, six.binary_type):
+ obj = obj.decode('utf-8')
+
          return obj

      def get_filtered(self, user_id):

Drew Freiberger (afreiberger) wrote :

ok, that patch actually didn't work. what I'm finding is that I'm getting about 1 in 50 page refreshes in Horizon passing a token without byte-encoding which is giving me false-positives.

The obj in that function above is:
{'id': '30339', 'name': 'testuser', 'email': '<email address hidden>', 'dn': 'uid=testuser,cn=users,dc=mysite,dc=com', 'enabled': True, 'options': {}}

It's actually the user_id being passed to this function that is the byte-encoded object.

Seyeong Kim (xtrusia) wrote :

FYI.

I was able to reproduce this issue in my local
and below code fixes this

inside token/token_formatters.py

https://review.opendev.org/#/c/663607/3/nova/virt/libvirt/storage/rbd_utils.py

but not sure it is proper for keystone as well?

Seyeong Kim (xtrusia) on 2019-06-12
tags: added: sts
Drew Freiberger (afreiberger) wrote :

  - Attempted to hack in the fix suggested by seyeong to cleanup byte-encoded strings, but
    getting only half fixed responses. half the actions still fail, I think this is because I placed
    this into attempt_convert_uuid_hex_to_bytes, rather than convert_uuid_hex_to_bytes.
    https://github.com/openstack/keystone/blob/master/keystone/token/token_formatters.py#L314,
  - It looks like there are a couple of contexts for assembling tokens
    that have domain_id being encapsulated through a different function,
    which may be part of the issue.
  - Dug through the ldap backend and found that there is a function that
    is supposed to take all the byte-coded data from ldap and turn it
    into strings and such for python usage:
    https://github.com/openstack/keystone/blob/master/keystone/identity/backends/ldap/common.py#L146
  - Tossing in debugging in this section of the code has resulted in
    showing everything coming out of the ldap result is pulled into
    python with proper formatting as all the variables get strained
    through a utf8_decode process when returned from ldap, so the issue
    is not in the ldap query response mangling.
    https://github.com/openstack/keystone/blob/master/keystone/identity/backends/ldap/common.py#L86
  - This leads me to believe the issue actually lies either in the
    database or in the code generating the tokens, which is why I'm most
    interested in the assemble routines in token/token_formatters.py.

Drew Freiberger (afreiberger) wrote :

Here's the patch I've been testing. It results in still having some failures to get project list.

https://pastebin.ubuntu.com/p/YC3KqH8WQJ/

Drew Freiberger (afreiberger) wrote :

Also tried this. seeing some amount of stripping happening in logs, for the user_id, but still unable to see projects drop-down in horizon when selected on secondary project membership.

Seyeong, if you're looking into reproducing, try adding a second project and make the user a Member of both projects, and then see after switching projects if you still see the projects drop-down.

*** token/token_formatters.py 2019-06-13 14:18:48.628706082 +0000
--- /home/ubuntu/token_formatters.py 2019-06-13 14:12:28.855950201 +0000
*************** class BasePayload(object):
*** 275,308 ****

          :param payload: this variant of payload
          :returns: a tuple of the payloads component data

          """
          raise NotImplementedError()

      @classmethod
      def convert_uuid_hex_to_bytes(cls, uuid_string):
          """Compress UUID formatted strings to bytes.

          :param uuid_string: uuid string to compress to bytes
          :returns: a byte representation of the uuid

          """
+ if isinstance(uuid_string, six.binary_type):
+ LOG.warning("DREW attempt_convert uuid_string: {}".format(uuid_string))
+ uuid_string = uuid_string.decode('utf-8')
+ LOG.warning("DREW attempt_convert uuid_string scrubbed: {}".format(uuid_string))
          uuid_obj = uuid.UUID(uuid_string)
          return uuid_obj.bytes

      @classmethod
      def convert_uuid_bytes_to_hex(cls, uuid_byte_string):
          """Generate uuid.hex format based on byte string.

          :param uuid_byte_string: uuid string to generate from
          :returns: uuid hex formatted string

          """
          uuid_obj = uuid.UUID(bytes=uuid_byte_string)
          return uuid_obj.hex

Drew Freiberger (afreiberger) wrote :

upgrading to field-critical.

The patch in place allows for some functionality but not enough for cloud to be usable by LDAP domain users.

Ryan Beisner (1chb1n) wrote :

This has an upstream bug element to it -- is that raised yet?

Changed in charm-keystone-ldap:
importance: Undecided → High
milestone: none → 19.07
Ryan Beisner (1chb1n) wrote :

Please attach a sanitized bundle.yaml as an example reproducer to this bug.

Changed in charm-keystone-ldap:
assignee: nobody → James Page (james-page)
James Page (james-page) wrote :

I feel this might be two bugs; one related to encoding of LDAP responses and the other due to behavioural changes - if that is the case we need to split them apart.

With regards to python3 encoding - the last three commits in master branch on token_formatters.py are related to py3 encoding - I think we need to pick those:

https://opendev.org/openstack/keystone/commit/79f468bad64c04a61e829029657fa6e905c610dd
https://opendev.org/openstack/keystone/commit/44c1b3d28407b66d2a854105f210d53f7cdd6b6f
https://opendev.org/openstack/keystone/commit/af3aef940c0162f12752a65282368d16c2d17c4f

One of the first two should cover the user_id encoding issue.

James Page (james-page) wrote :

The commits picked may not go far enough as they just seem to touch the Federated token formatter classes - may need to expand that to cover other Payload types.

James Page (james-page) wrote :

Tl;DR - I think the disassemble functions need to deal with the encoding better under py3; if the user_id gets into keystone decoded, then it should be dealt with correctly throughout.

James Page (james-page) wrote :

Raising a bug task against keystone as I think that we may need to expanded the decode coverage in token_formatters.

Changed in keystone (Ubuntu):
assignee: nobody → James Page (james-page)
importance: Undecided → High
status: New → In Progress
Changed in charm-keystone-ldap:
status: New → Invalid
James Page (james-page) wrote :

Marking charm task as invalid - the py3 encoding issues are in keystone itself, not the deployment tooling.

James Page (james-page) wrote :

Behavioural bug reference bug 1832766

summary: - keystone LDAP integration in rocky not working for RBAC rules or token
- auth
+ py3: inconsistent encoding of token fields
Drew Freiberger (afreiberger) wrote :

Positive outcome for initial testing of the ppa:james-page/rocky patch. contacting other users of this cloud for confirmation.

Fix proposed to branch: master
Review: https://review.opendev.org/665617

Changed in keystone:
assignee: nobody → James Page (james-page)
status: New → In Progress
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package keystone - 2:16.0.0~b1~git2019061326.4f1667679-0ubuntu2

---------------
keystone (2:16.0.0~b1~git2019061326.4f1667679-0ubuntu2) eoan; urgency=medium

  * d/p/token-consistently-decode-binary-types.patch: Ensure binary
    types are consistently decoded under Python 3 (LP: #1832265).

 -- James Page <email address hidden> Fri, 21 Jun 2019 05:32:39 +0100

Changed in keystone (Ubuntu):
status: In Progress → Fix Released
James Page (james-page) on 2019-06-21
Changed in keystone (Ubuntu Cosmic):
status: New → Triaged
importance: Undecided → High
Changed in keystone (Ubuntu Disco):
importance: Undecided → High
status: New → Triaged
Dmitrii Shcherbakov (dmitriis) wrote :

Ran into a related problem during debugging of dashboard errors ("Unable to retrieve key pairs") with a Rocky cloud & identity federation.

There was no clear indication as to why failures occurred.

https://paste.ubuntu.com/p/v5HXyyWXC2/ (full pdb trace)

At a high level I was getting validation failures for the identity provider (which was enabled in Keystone and was otherwise correct in terms of config) in the /v3/auth/token code path.

I narrowed it down to a validation error due to a type mismatch (bytes vs str):

1) the error occurs in send_notification:

> /usr/lib/python3/dist-packages/keystone/auth/plugins/mapped.py(101)handle_scoped_token()->None
-> send_notification(taxonomy.OUTCOME_SUCCESS)
(Pdb) l
 96 # send off failed authentication notification, raise the exception
 97 # after sending the notification
 98 send_notification(taxonomy.OUTCOME_FAILURE)
 99 raise
100 else:
101 -> send_notification(taxonomy.OUTCOME_SUCCESS)

# ...

2) this is how the validation error looks like:

(Pdb) setattr(self, FED_CRED_KEYNAME_IDENTITY_PROVIDER, identity_provider)
*** ValueError: identity_provider failed validation: <function FederatedCredential.<lambda> at 0x7fa0016ef9d8>

3) the lambda function where the error occurs

 67 class FederatedCredential(Credential):
 68 identity_provider = cadftype.ValidatorDescriptor(
 69 FED_CRED_KEYNAME_IDENTITY_PROVIDER,
 70 -> lambda x: isinstance(x, six.string_types))
 71 user = cadftype.ValidatorDescriptor(
 72 FED_CRED_KEYNAME_USER,
 73 lambda x: isinstance(x, six.string_types))
 74 groups = cadftype.ValidatorDescriptor(
 75 FED_CRED_KEYNAME_GROUPS,

4) type comparison (b'adfs' is the identity provider name):

((Pdb)) x
b'adfs'
((Pdb)) six.string_types
(<class 'str'>,)
((Pdb)) type(x)
<class 'bytes'>

Using a package from James' PPA helped as I am not getting errors in the same code-path anymore.

apt policy keystone
keystone:
  Installed: 2:14.1.0-0ubuntu2~ubuntu18.04.1~ppa201906140719
  Candidate: 2:14.1.0-0ubuntu2~ubuntu18.04.1~ppa201906140719
  Version table:
 *** 2:14.1.0-0ubuntu2~ubuntu18.04.1~ppa201906140719 500

When clicking through tabs very fast I encountered a glitch which results in the following error messages being displayed (see the screencast in the attachment):

Error: "Unable to retrieve key pairs"/"Unable to retrieve images"/""Unable to retrieve server groups"
Warning: "Policy check failed"

I tried to set breakpoints in the same place - the same validation error does NOT occur with the patch so this is something else unrelated to py2 vs py3 string handling.

James Page (james-page) wrote :

Ubuntu SRU information:

[Impact]
Due to inconsistent decode/encode of bytestrings under py3, keystone ldap integration is broken when keystone is run under Python 3.

[Test Case]
Deploy keystone
Configure to use LDAP

[Regression Potential]
The proposed patch has been +2'ed by upstream and validated as resolving this issue by the original bug reporter; the change simply ensures that any encoded values are decoded before use.

Changed in keystone (Ubuntu Cosmic):
status: Triaged → Won't Fix

Hello Drew, or anyone else affected,

Accepted keystone into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/keystone/2:15.0.0-0ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in keystone (Ubuntu Disco):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-disco
Corey Bryant (corey.bryant) wrote :

Hello Drew, or anyone else affected,

Accepted keystone into stein-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:stein-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-stein-needed to verification-stein-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-stein-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-stein-needed
James Page (james-page) wrote :

Hello Drew, or anyone else affected,

Accepted keystone into rocky-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:rocky-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-rocky-needed to verification-rocky-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-rocky-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-rocky-needed
tom king (kingttx) wrote :

James, I have the same issue. Which package do you want the version on? We have a PoC on Rocky using this charm and have this exact issue.

Do we put this repository on the controllers or everywhere (compute nodes, ceph, etc.)?

Thank you.

Drew Freiberger (afreiberger) wrote :

@kingttx You will only need to add this to all the keystone charm's units. Then run apt-get upgrade to install the updated keystone* packages, and lastly run 'service apache2 restart' in order to reload the wsgi scripts.

We have tested both bionic-rocky-proposed and bionic-stein-proposed with success.

Frode Nordahl (fnordahl) on 2019-08-13
Changed in charm-keystone-ldap:
milestone: 19.07 → none
tom king (kingttx) wrote :

@Drew Freiberger - that fixed my issue as well. I did have some problem with 'Empty Response' for a few minutes after updating the packages and restarting apache on the three Keystone nodes, but it finally started working and the Horizon issues seemed to be fixed.

Now on to the next error.

Thank you, folks!
Tom

Corey Bryant (corey.bryant) wrote :

@Drew/@Tom, If you've successfully tested disco-proposed, rocky-proposed or stein-proposed on bionic, can you tag the bug accordingly with verification-* tags? Thanks, Corey

Drew Freiberger (afreiberger) wrote :

I've updated the rocky/stein tags to verificiation-rocky/stein-done as requested per my comment #36.

tags: added: verification-done-rocky verification-done-stein
removed: verification-rocky-needed verification-stein-needed
tags: added: verification-rocky-done verification-stein-done
removed: verification-done-rocky verification-done-stein
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers