failed to add a new compute node after upgrading undercloud from newton to master, the overcloud nodes(controllers and existing compute nodes are still in newton).
2018-01-24 04:08:48.324 31303 DEBUG heat.engine.scheduler [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Task create from TemplateResource "1" Stack "overcloud-Compute-7tlf5h3fnqpl" [28a3ee6e-4177-4c63-b296-58bc6a4d37a5] running step /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:209
2018-01-24 04:08:48.327 31303 DEBUG heat.engine.scheduler [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Task update from ResourceGroup "Compute" [28a3ee6e-4177-4c63-b296-58bc6a4d37a5] Stack "overcloud" [08176345-dce2-44e4-9727-638fcc9014c1] running step /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:209
2018-01-24 04:08:48.327 31303 DEBUG heat.engine.scheduler [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Task _run_to_completion from ResourceGroup "Compute" [28a3ee6e-4177-4c63-b296-58bc6a4d37a5] Stack "overcloud" [08176345-dce2-44e4-9727-638fcc9014c1] running step /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:209
2018-01-24 04:08:48.333 31303 DEBUG heat.engine.scheduler [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Task update from ResourceGroup "Compute" [28a3ee6e-4177-4c63-b296-58bc6a4d37a5] Stack "overcloud" [08176345-dce2-44e4-9727-638fcc9014c1] sleeping _sleep /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:150
2018-01-24 04:08:48.346 31303 DEBUG heat.engine.scheduler [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Task create from TemplateResource "1" Stack "overcloud-Compute-7tlf5h3fnqpl" [28a3ee6e-4177-4c63-b296-58bc6a4d37a5] complete step /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:215
2018-01-24 04:08:48.354 31303 DEBUG heat.engine.sync_point [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] [28a3ee6e-4177-4c63-b296-58bc6a4d37a5:0217f520-29fc-49f5-9b25-28a617227125:True] Ready 28a3ee6e-4177-4c63-b296-58bc6a4d37a5: Got (634, True), (284, False) sync /usr/lib/python2.7/site-packages/heat/engine/sync_point.py:145
2018-01-24 04:08:48.354 31303 INFO heat.engine.stack [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] [overcloud-Compute-7tlf5h3fnqpl(28a3ee6e-4177-4c63-b296-58bc6a4d37a5)] update traversal 0217f520-29fc-49f5-9b25-28a617227125 complete
2018-01-24 04:08:48.355 31303 INFO heat.engine.stack [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Stack UPDATE COMPLETE (overcloud-Compute-7tlf5h3fnqpl): Stack UPDATE completed successfully
2018-01-24 04:08:48.359 31303 DEBUG heat.engine.stack [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Persisting stack overcloud-Compute-7tlf5h3fnqpl status UPDATE COMPLETE _send_notification_and_add_event /usr/lib/python2.7/site-packages/heat/engine/stack.py:1019
2018-01-24 04:08:49.334 31303 DEBUG heat.engine.scheduler [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Task update from ResourceGroup "Compute" [28a3ee6e-4177-4c63-b296-58bc6a4d37a5] Stack "overcloud" [08176345-dce2-44e4-9727-638fcc9014c1] running step /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:209
2018-01-24 04:08:49.335 31303 DEBUG heat.engine.scheduler [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Task _run_to_completion from ResourceGroup "Compute" [28a3ee6e-4177-4c63-b296-58bc6a4d37a5] Stack "overcloud" [08176345-dce2-44e4-9727-638fcc9014c1] running step /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:209
2018-01-24 04:08:49.343 31303 DEBUG heat.engine.scheduler [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Task _run_to_completion from ResourceGroup "Compute" [28a3ee6e-4177-4c63-b296-58bc6a4d37a5] Stack "overcloud" [08176345-dce2-44e4-9727-638fcc9014c1] complete step /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:215
2018-01-24 04:08:49.410 31303 DEBUG heat.engine.scheduler [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Task update from ResourceGroup "Compute" [28a3ee6e-4177-4c63-b296-58bc6a4d37a5] Stack "overcloud" [08176345-dce2-44e4-9727-638fcc9014c1] complete step /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:215
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Unexpected exception in resource check.: NotFound: Member '2' not found in group resource 'Compute'.
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource Traceback (most recent call last):
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource File "/usr/lib/python2.7/site-packages/heat/engine/check_resource.py", line 321, in check
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource rsrc, stack)
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource File "/usr/lib/python2.7/site-packages/heat/engine/check_resource.py", line 253, in _initiate_propagate_resource
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource input_data = _get_input_data(req_node, input_forward_data)
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource File "/usr/lib/python2.7/site-packages/heat/engine/check_resource.py", line 236, in _get_input_data
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource return rsrc.node_data().as_dict()
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource File "/usr/lib/python2.7/site-packages/heat/engine/node_data.py", line 85, in as_dict
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource raise v
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource NotFound: Member '2' not found in group resource 'Compute'.
2018-01-24 04:08:49.913 31303 ERROR heat.engine.check_resource
2018-01-24 04:08:49.914 31303 INFO heat.engine.stack [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Stack UPDATE FAILED (overcloud): Member '2' not found in group resource 'Compute'.
2018-01-24 04:08:49.924 31303 DEBUG heat.engine.stack [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Persisting stack overcloud status UPDATE FAILED _send_notification_and_add_event /usr/lib/python2.7/site-packages/heat/engine/stack.py:1019
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker [req-43a18dc4-453a-45d3-8e31-241ad2da622a - admin - default default] Unhandled exception in check_resource: NotFound: Member '2' not found in group resource 'Compute'.
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker Traceback (most recent call last):
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker File "/usr/lib/python2.7/site-packages/heat/engine/worker.py", line 45, in exception_wrapper
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker return func(*args, **kwargs)
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker File "/usr/lib/python2.7/site-packages/heat/engine/worker.py", line 196, in check_resource
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker is_update, adopt_stack_data, rsrc, stack)
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker File "/usr/lib/python2.7/site-packages/heat/engine/check_resource.py", line 327, in check
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker stack, msg)
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker self.force_reraise()
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker six.reraise(self.type_, self.value, self.tb)
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker File "/usr/lib/python2.7/site-packages/heat/engine/check_resource.py", line 321, in check
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker rsrc, stack)
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker File "/usr/lib/python2.7/site-packages/heat/engine/check_resource.py", line 253, in _initiate_propagate_resource
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker input_data = _get_input_data(req_node, input_forward_data)
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker File "/usr/lib/python2.7/site-packages/heat/engine/check_resource.py", line 236, in _get_input_data
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker return rsrc.node_data().as_dict()
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker File "/usr/lib/python2.7/site-packages/heat/engine/node_data.py", line 85, in as_dict
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker raise v
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker NotFound: Member '2' not found in group resource 'Compute'.
2018-01-24 04:08:49.962 31303 ERROR heat.engine.worker
stack output:
[...]
2018-01-24 03:33:08Z [overcloud-Compute-7tlf5h3fnqpl.1.NovaCompute]: CREATE_COMPLETE state changed
2018-01-24 03:33:08Z [overcloud-Compute-7tlf5h3fnqpl.1.TenantPort]: CREATE_IN_PROGRESS state changed
2018-01-24 03:33:08Z [overcloud-Compute-7tlf5h3fnqpl.1.ExternalPort]: CREATE_IN_PROGRESS state changed
2018-01-24 03:33:08Z [overcloud-Compute-7tlf5h3fnqpl.1.StorageMgmtPort]: CREATE_IN_PROGRESS state changed
2018-01-24 03:33:08Z [overcloud-Compute-7tlf5h3fnqpl.1.ManagementPort]: CREATE_IN_PROGRESS state changed
2018-01-24 03:33:08Z [overcloud-Compute-7tlf5h3fnqpl.1.InternalApiPort]: CREATE_IN_PROGRESS state changed
2018-01-24 03:33:08Z [overcloud-Compute-7tlf5h3fnqpl.1.StoragePort]: CREATE_IN_PROGRESS state changed
2018-01-24 03:33:09Z [overcloud-Compute-7tlf5h3fnqpl.1.StorageMgmtPort]: CREATE_COMPLETE state changed
2018-01-24 03:33:10Z [overcloud-Compute-7tlf5h3fnqpl.1.ManagementPort]: CREATE_COMPLETE state changed
2018-01-24 03:33:10Z [overcloud-Compute-7tlf5h3fnqpl.1.ExternalPort]: CREATE_COMPLETE state changed
2018-01-24 03:33:14Z [overcloud-Compute-7tlf5h3fnqpl.1.StoragePort]: CREATE_COMPLETE state changed
2018-01-24 03:33:14Z [overcloud-Compute-7tlf5h3fnqpl.1.InternalApiPort]: CREATE_COMPLETE state changed
2018-01-24 03:33:15Z [overcloud-Compute-7tlf5h3fnqpl.1.TenantPort]: CREATE_COMPLETE state changed
2018-01-24 03:33:15Z [overcloud-Compute-7tlf5h3fnqpl.1.NetworkConfig]: CREATE_IN_PROGRESS state changed
2018-01-24 03:33:15Z [overcloud-Compute-7tlf5h3fnqpl.1.NetIpMap]: CREATE_IN_PROGRESS state changed
2018-01-24 03:33:17Z [overcloud-Compute-7tlf5h3fnqpl.1.NetIpMap]: CREATE_COMPLETE state changed
2018-01-24 03:33:17Z [overcloud-Compute-7tlf5h3fnqpl.1.NovaComputeConfig]: CREATE_IN_PROGRESS state changed
2018-01-24 03:33:17Z [overcloud-Compute-7tlf5h3fnqpl.1.NetworkConfig]: CREATE_COMPLETE state changed
2018-01-24 03:33:17Z [overcloud-Compute-7tlf5h3fnqpl.1.NovaComputeConfig]: CREATE_COMPLETE state changed
2018-01-24 03:33:17Z [overcloud-Compute-7tlf5h3fnqpl.1.NetworkDeployment]: CREATE_IN_PROGRESS state changed
2018-01-24 04:08:11Z [overcloud-Compute-7tlf5h3fnqpl.1.NetworkDeployment]: SIGNAL_IN_PROGRESS Signal: deployment 6f2e6a48-25f1-4745-aaaf-150ef651e8d9 succeeded
2018-01-24 04:08:12Z [overcloud-Compute-7tlf5h3fnqpl.1.NetworkDeployment]: CREATE_COMPLETE state changed
2018-01-24 04:08:12Z [overcloud-Compute-7tlf5h3fnqpl.1.UpdateDeployment]: CREATE_IN_PROGRESS state changed
2018-01-24 04:08:19Z [overcloud-Compute-7tlf5h3fnqpl.1.UpdateDeployment]: SIGNAL_IN_PROGRESS Signal: deployment 5ae2ff3f-6336-4f47-a187-25d25bb9fc96 succeeded
2018-01-24 04:08:19Z [overcloud-Compute-7tlf5h3fnqpl.1.UpdateDeployment]: CREATE_COMPLETE state changed
2018-01-24 04:08:19Z [overcloud-Compute-7tlf5h3fnqpl.1.NovaComputeDeployment]: CREATE_IN_PROGRESS state changed
2018-01-24 04:08:32Z [overcloud-Compute-7tlf5h3fnqpl.1.NovaComputeDeployment]: SIGNAL_IN_PROGRESS Signal: deployment 6fc2744b-5143-43c4-b312-215b86bf8d5f succeeded
2018-01-24 04:08:33Z [overcloud-Compute-7tlf5h3fnqpl.1.NovaComputeDeployment]: CREATE_COMPLETE state changed
2018-01-24 04:08:33Z [overcloud-Compute-7tlf5h3fnqpl.1.NodeTLSCAData]: CREATE_IN_PROGRESS state changed
2018-01-24 04:08:33Z [overcloud-Compute-7tlf5h3fnqpl.1.ComputeExtraConfigPre]: CREATE_IN_PROGRESS state changed
2018-01-24 04:08:33Z [overcloud-Compute-7tlf5h3fnqpl.1.NodeTLSCAData]: CREATE_COMPLETE state changed
2018-01-24 04:08:33Z [overcloud-Compute-7tlf5h3fnqpl.1.SshHostPubKey]: CREATE_IN_PROGRESS state changed
2018-01-24 04:08:35Z [overcloud-Compute-7tlf5h3fnqpl.1.ComputeExtraConfigPre]: CREATE_COMPLETE state changed
2018-01-24 04:08:35Z [overcloud-Compute-7tlf5h3fnqpl.1.NodeExtraConfig]: CREATE_IN_PROGRESS state changed
2018-01-24 04:08:36Z [overcloud-Compute-7tlf5h3fnqpl.1.NodeExtraConfig]: CREATE_COMPLETE state changed
2018-01-24 04:08:47Z [overcloud-Compute-7tlf5h3fnqpl.1.SshHostPubKey]: CREATE_COMPLETE state changed
2018-01-24 04:08:47Z [overcloud-Compute-7tlf5h3fnqpl.1]: CREATE_COMPLETE Stack CREATE completed successfully
2018-01-24 04:08:48Z [overcloud-Compute-7tlf5h3fnqpl.1]: CREATE_COMPLETE state changed
2018-01-24 04:08:48Z [overcloud-Compute-7tlf5h3fnqpl]: UPDATE_COMPLETE Stack UPDATE completed successfully
2018-01-24 04:08:49Z [Compute]: UPDATE_COMPLETE state changed
2018-01-24 04:08:49Z [overcloud]: UPDATE_FAILED Member '2' not found in group resource 'Compute'.
