There is an issue with the manifest apply for openstack::keystone::endpoint::runtime sysinv 2020-05-31 23:19:49.499 221876 INFO sysinv.conductor.manager [-] Setting config target of host 'controller-1' to '62042df4-8e89-4d13-b82c-7e2afd09c913'. sysinv 2020-05-31 23:19:49.540 221876 WARNING sysinv.conductor.manager [-] controller-1: iconfig out of date: target 62042df4-8e89-4d13-b82c-7e2afd09c913, applied 7e257187-e75e-46bf-8165-20066d02242f sysinv 2020-05-31 23:19:49.542 221876 WARNING sysinv.conductor.manager [-] SYS_I Raise system config alarm: host controller-1 config applied: 7e257187-e75e-46bf-8165-20066d02242f vs. target: 62042df4-8e89-4d13-b82c-7e2afd09c913. sysinv 2020-05-31 23:19:49.598 221876 INFO sysinv.conductor.manager [-] _config_update_hosts config_uuid=62042df4-8e89-4d13-b82c-7e2afd09c913 sysinv 2020-05-31 23:19:49.599 221876 INFO sysinv.conductor.manager [-] applying runtime manifest config_uuid=62042df4-8e89-4d13-b82c-7e2afd09c913, classes: ['openstack::keystone::endpoint::runtime', 'platform::firewall::runtime'] sysinv 2020-05-31 23:19:49.637 221876 INFO sysinv.puppet.puppet [-] Updating hiera for host: controller-1 with config_uuid: 62042df4-8e89-4d13-b82c-7e2afd09c913 sysinv 2020-05-31 23:19:53.748 221876 INFO sysinv.puppet.kubernetes [-] get_kubernetes_join_cmd join_cmd=kubeadm join 192.168.205.1:6443 --token xjtepg.prb4umxjlu2yl9dt --discovery-token-ca-cert-hash sha256:967dc2ea50af4679ed43441556d52bcb9f7fb168dd9188bd00e18611149d38a5 --control-plane --certificate-key b08e693f9a388d46c7021c15fb34025f8f6cef5ff5a43f71d57aae7ca760a5ac --cri-socket /var/run/containerd/containerd.sock sysinv 2020-05-31 23:19:54.657 222753 INFO sysinv.api.controllers.v1.host [-] Provisioned storage node(s) [] sysinv 2020-05-31 23:19:55.132 221876 INFO sysinv.helm.helm [-] HelmOperator: Loading available helm and armada plugins. sysinv 2020-05-31 23:19:55.135 221876 INFO sysinv.helm.helm [-] Plugins for generic : loaded from sysinv - /usr/lib64/python2.7/site-packages. sysinv 2020-05-31 23:19:55.135 221876 INFO sysinv.helm.helm [-] Plugins for stx-monitor : loaded from sysinv - /usr/lib64/python2.7/site-packages. sysinv 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet [-] failed to create system config: KeyError: 'keystone' 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet Traceback (most recent call last): 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet File "/usr/lib64/python2.7/site-packages/sysinv/puppet/puppet.py", line 116, in update_system_config 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet config.update(puppet_plugin.obj.get_system_config()) 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet File "/usr/lib64/python2.7/site-packages/sysinv/puppet/nfv.py", line 96, in get_system_config 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet keystone_auth_data = helm_data.get_keystone_auth_data() 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet File "/usr/lib64/python2.7/site-packages/sysinv/helm/helm.py", line 49, in _wrapper 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet return func(self, *args, **kwargs) 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet File "/usr/lib64/python2.7/site-packages/sysinv/helm/helm.py", line 845, in get_keystone_auth_data 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet keystone_operator = self.chart_operators[self.HELM_CHART_KEYSTONE] 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet KeyError: 'keystone' 2020-05-31 23:19:55.136 221876 ERROR sysinv.puppet.puppet sysinv 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task [-] Error during ConductorManager._conductor_audit: 'keystone': KeyError: 'keystone' 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task Traceback (most recent call last): 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/periodic_task.py", line 180, in run_periodic_tasks 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task task(self, context) 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 4937, in _conductor_audit 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task self._controller_config_active_apply(context) 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 4666, in _controller_config_active_apply 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task context, config_uuid, config_dict) 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 8626, in _config_apply_runtime_manifest 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task force=force) 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 8572, in _config_update_puppet 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task self._puppet.update_system_config() 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task File "/usr/lib64/python2.7/site-packages/sysinv/puppet/puppet.py", line 31, in _wrapper 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task func(self, *args, **kwargs) 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task File "/usr/lib64/python2.7/site-packages/sysinv/puppet/puppet.py", line 116, in update_system_config 020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task config.update(puppet_plugin.obj.get_system_config()) 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task File "/usr/lib64/python2.7/site-packages/sysinv/puppet/nfv.py", line 96, in get_system_config 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task keystone_auth_data = helm_data.get_keystone_auth_data() 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task File "/usr/lib64/python2.7/site-packages/sysinv/helm/helm.py", line 49, in _wrapper 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task return func(self, *args, **kwargs) 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task File "/usr/lib64/python2.7/site-packages/sysinv/helm/helm.py", line 845, in get_keystone_auth_data 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task keystone_operator = self.chart_operators[self.HELM_CHART_KEYSTONE] 2020-05-31 23:19:55.139 221876 ERROR sysinv.openstack.common.periodic_task KeyError: 'keystone'