Vcenter-as-compute: 16-04/newton: provisioning fails handling container vcenter-plugin

[Sarath]

Version: 4.0.1.0-48-newton
Topology: 3node HA with multiple computes (multi-cluster Esxi) & Kvm

in one of cluster container up but not vcenter-plugin service.
in other cluster, container not up

root@5a10s26:~# tailf /var/log/syslog
Sep 5 15:03:45 5a10s26 puppet-agent[2462]: Skipping run of Puppet configuration client; administratively disabled (Reason: 'Disabled by default on new or unconfigured old installations');
Sep 5 15:03:45 5a10s26 puppet-agent[2462]: Use 'puppet agent --enable' to re-enable.
Sep 5 15:05:01 5a10s26 CRON[19315]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Sep 5 15:15:01 5a10s26 CRON[19428]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Sep 5 15:17:01 5a10s26 CRON[19431]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Sep 5 15:17:59 5a10s26 dockerd[17242]: time="2017-09-05T15:17:59.338506033-07:00" level=error msg="Error setting up exec command in container vcplugin: No such container: vcplugin"
Sep 5 15:17:59 5a10s26 dockerd[17242]: time="2017-09-05T15:17:59.338586465-07:00" level=error msg="Handler for POST /v1.25/containers/vcplugin/exec returned error: No such container: vcplugin"
Sep 5 15:18:02 5a10s26 dockerd[17242]: time="2017-09-05T15:18:02.479963757-07:00" level=error msg="Error setting up exec command in container vcplugin: No such container: vcplugin"
Sep 5 15:18:02 5a10s26 dockerd[17242]: time="2017-09-05T15:18:02.480016292-07:00" level=error msg="Handler for POST /v1.25/containers/vcplugin/exec returned error: No such container: vcplugin"
Sep 5 15:25:01 5a10s26 CRON[19613]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)

[kamlesh parmar (kparmar)]

This is nothing to do with vcenter.

The other configured roles for this node (contrail-lb, vcenter-compute) didnt provision as well.

The hostname for the node failing to launch vcplugin is same as the SM itself. This causes the ssl certs task to fail on this node:
"2017-09-05 13:32:12,626-INFO-sm_ansible_callback.py:43-append(): fatal: [10.87.36.27] (item={u'dest': u'/etc/contrailctl/ssl/server.pem', u'src': u'/etc/contrail_smgr/puppet/ssl/5a10s26.pem'}) => {"failed": true, "item": {"dest": "/etc/contrailctl/ssl/server.pem", "src": "/etc/contrail_smgr/puppet/ssl/5a10s26.pem"}, "msg": "Unable to find '/etc/contrail_smgr/puppet/ssl/5a10s26.pem' in expected paths."}"
"2017-09-05 13:32:12,629-INFO-sm_ansible_callback.py:43-append(): skipping: [10.87.36.18]"
"2017-09-05 13:32:12,631-INFO-sm_ansible_callback.py:43-append(): fatal: [10.87.36.27]: FAILED! => (item - None) {"changed": false, "failed": true, "msg": "One or more items failed"}"
"2017-09-05 13:32:12,632-INFO-sm_ansible_callback.py:43-append(): fatal: [10.87.36.27] (item={u'dest': u'/etc/contrailctl/ssl/server-privkey.pem', u'src': u'/etc/contrail_smgr/puppet/ssl/5a10s26-privkey.pem'}) => {"failed": true, "item": {"dest": "/etc/contrailctl/ssl/server-privkey.pem", "src": "/etc/contrail_smgr/puppet/ssl/5a10s26-privkey.pem"}, "msg": "Unable to find '/etc/contrail_smgr/puppet/ssl/5a10s26-privkey.pem' in expected paths."}"

[Sarath (nsarath)]

As re-image only target-nodes and re-provisioning from SM handled hostname correctly,
need to figure out where this hostname change happened.

Will monitor during next complete fresh install of both target-nodes and SM node.

So, keeping the Bug priority & state accordingly.