B&R with stx-openstack: Unlock failed after restore playbook was run

[Dan Voiculeasa]

Brief Description
-----------------
Restoring a system that had stx-openstack applied prior to the backup will fail.
Failure at unlocking controller-0. Root cause: no plugins detected.

Severity
--------
Critical: System/Feature is not usable due to the defect

Steps to Reproduce
------------------
Do B&R on a system with stx-openstack applied.

Expected Behavior
------------------
Controller can be unlocked after restore_platform playbook is run.

Actual Behavior
----------------
Failed to unlock.

Reproducibility
---------------
100% reproducible

System Configuration
--------------------
AIO-SX ipv4, but this should be seen on all types

Branch/Pull Time/Commit
-----------------------
4 Aug 2020

Last Pass
---------
May/June, maybe more recent.

Timestamp/Logs
--------------
  File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/amqp.py", line 437, in process_data
    **args)
  File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/dispatcher.py", line 172, in dispatch
    result = getattr(proxyobj, method)(ctxt, **kwargs)
  File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 1750, in configure_ihost
    self._puppet.update_system_config()
  File "/usr/lib64/python2.7/site-packages/sysinv/puppet/puppet.py", line 31, in _wrapper
    func(self, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/sysinv/puppet/puppet.py", line 116, in update_system_config
    config.update(puppet_plugin.obj.get_system_config())
  File "/usr/lib64/python2.7/site-packages/sysinv/puppet/nfv.py", line 96, in get_system_config
    keystone_auth_data = helm_data.get_keystone_auth_data()
  File "/usr/lib64/python2.7/site-packages/sysinv/helm/helm.py", line 50, in _wrapper
    return func(self, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/sysinv/helm/helm.py", line 864, in et_keystone_auth_data
    keystone_operator = self.chart_operators[self.HELM_CHART_KEYSTONE]
KeyError: 'keystone'

Test Activity
-------------
Developer Testing

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/745332

[Ghada Khalil (gkhalil)]

As per Frank Miller, this is also an issue with stx.4.0, so marking as gating for both releases.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/745332
Committed: https://git.openstack.org/cgit/starlingx/config/commit/?id=b9013aaea5f597d3fb55c37a9fada2e4f8877a81
Submitter: Zuul
Branch: master

commit b9013aaea5f597d3fb55c37a9fada2e4f8877a81
Author: Dan Voiculeasa <email address hidden>
Date: Fri Aug 7 13:22:21 2020 +0300

    Fix unlock after restore with openstack applied

    Unlock will fail if plugins are not enabled when generating a puppet
    system config if stx-openstack was applied. This is observed during
    restore procedure.

    Activate plugins before updating puppet system config.
    Puppet system config when openstack is applied depends on helm plugins
    to be available.

    Closes-Bug: 1890447
    Change-Id: Ib12d9b8d1f477ee88ac7c1a1e3f05a2e9de2d4af
    Signed-off-by: Dan Voiculeasa <email address hidden>