keystone upgrade from queens to rocky didnt install python3-ldappool

[Alexander Balderson]

Working through an upgrade from queens to rocky, while upgrading neutron-api, the keystone leader unit went into an error state during the identity-service-relation-changed hook with a bunch of 500 errors:

2020-04-17 19:44:44 ERROR juju-log identity-service:86: The call within manager.py failed with the error: 'An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-9731ac41-d36e-4d66-9721-e8fa50689e57)'. The call was: path=['user_exists'], args=('neutron',), kwargs={'domain': 'default'}, api_version=None

Before it went into an error state; the charm message was "updating endpoint neutron"

The keystone log showed that it was failing to import ldappool:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/keystone/common/wsgi.py", line 148, in __call__
    result = method(req, **params)
  File "/usr/lib/python3/dist-packages/keystone/auth/controllers.py", line 67, in authenticate_for_token
    self.authenticate(request, auth_info, auth_context)
  File "/usr/lib/python3/dist-packages/keystone/auth/controllers.py", line 236, in authenticate
    auth_info.get_method_data(method_name))
  File "/usr/lib/python3/dist-packages/keystone/auth/plugins/password.py", line 31, in authenticate
    user_info = auth_plugins.UserAuthInfo.create(auth_payload, METHOD_NAME)
  File "/usr/lib/python3/dist-packages/keystone/auth/plugins/core.py", line 102, in create
    user_auth_info._validate_and_normalize_auth_data(auth_payload)
  File "/usr/lib/python3/dist-packages/keystone/auth/plugins/core.py", line 189, in _validate_and_normalize_auth_data
    auth_payload)
  File "/usr/lib/python3/dist-packages/keystone/auth/plugins/core.py", line 166, in _validate_and_normalize_auth_data
    user_name, domain_ref['id'])
  File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 116, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/keystone/identity/core.py", line 415, in wrapper
    self.driver, PROVIDERS.resource_api)
  File "/usr/lib/python3/dist-packages/keystone/identity/core.py", line 309, in setup_domain_drivers
    resource_api)
  File "/usr/lib/python3/dist-packages/keystone/identity/core.py", line 164, in _setup_domain_drivers_from_files
    -len(DOMAIN_CONF_FTAIL)])
  File "/usr/lib/python3/dist-packages/keystone/identity/core.py", line 130, in _load_config_from_file
    domain_config['driver'] = self._load_driver(domain_config)
  File "/usr/lib/python3/dist-packages/keystone/identity/core.py", line 93, in _load_driver
    domain_config['cfg'])
  File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 77, in load_driver
    invoke_args=args)
  File "/usr/lib/python3/dist-packages/stevedore/driver.py", line 61, in __init__
    warn_on_missing_entrypoint=warn_on_missing_entrypoint
  File "/usr/lib/python3/dist-packages/stevedore/named.py", line 81, in __init__
    verify_requirements)
  File "/usr/lib/python3/dist-packages/stevedore/extension.py", line 203, in _load_plugins
    self._on_load_failure_callback(self, ep, err)
  File "/usr/lib/python3/dist-packages/stevedore/extension.py", line 195, in _load_plugins
    verify_requirements,
  File "/usr/lib/python3/dist-packages/stevedore/named.py", line 158, in _load_one_plugin
    verify_requirements,
  File "/usr/lib/python3/dist-packages/stevedore/extension.py", line 223, in _load_one_plugin
    plugin = ep.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/__init__.py", line 13, in <module>
    from keystone.identity.backends.ldap.core import * # noqa
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/core.py", line 26, in <module>
    from keystone.identity.backends.ldap import common as common_ldap
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/common.py", line 25, in <module>
    import ldappool
ModuleNotFoundError: No module named 'ldappool'
(keystone.common.wsgi): 2020-04-17 19:57:03,056 ERROR No module named 'ldappool'

And i was unable to import ldappool in python3, but was able to in python2. I manually installed python3-ldappool to match the note in the keystone-ldap package:
https://github.com/openstack/charm-keystone-ldap/blob/4163fdcbeefbe197204ae06177e81d776baa548e/src/lib/charm/openstack/keystone_ldap.py#L180

After re-running the hook; the charm came back up.
Bundle attached; crashdump to follow.

[Alexander Balderson (asbalderson)]

the upgrade order so far has been:
keystone
ceph-mon
ceph-osd
ceph-radosgw
glance
nova-cloud-controller
neutron-api

[Alexander Balderson (asbalderson)]

ldap overlay

[Billy Olsen (billy-olsen)]

This is a duplicate of bug https://bugs.launchpad.net/charm-keystone-ldap/+bug/1806111, which was fixed in the 21.04 release of the charms. A change was made to have the subordinate charms pass package information over the subordinate relation in order to allow the principal charm to update relevant packages at upgrade time.