vcenter-as-compute: SM provisioning fails @ ansible TASK : Copy cert files onto container

[Sarath]

Version: 4.0.1.0-34-mitaka

Debugged the issue with Dev Kamlesh and he knows the problem

Below seen during SM provisoning later part after vcenter related succeeds,

"2017-08-18 17:28:06,520-INFO-sm_ansible_callback.py:43-append(): TASK [node : Copy SSL CA Cert file for use by container]"
"2017-08-18 17:28:07,188-INFO-sm_ansible_callback.py:43-append(): changed: [10.87.36.19] => (item={u'dest': u'/etc/contrail/ssl/certs/ca-cert.pem', u'src': u'/etc/contrail_smgr/puppet/ssl/ca-cert.pem'})"
"2017-08-18 17:28:07,191-INFO-sm_ansible_callback.py:43-append(): changed: [10.87.36.19]"
"2017-08-18 17:28:07,203-INFO-sm_ansible_callback.py:43-append(): TASK [node : Copy cert files onto container]"
"2017-08-18 17:28:07,384-INFO-sm_ansible_callback.py:43-append(): fatal: [10.87.36.19] (item={u'dest': u'/etc/contrail/ssl/certs/server.pem', u'src': u'/etc/contrail_smgr/puppet/ssl/ContrailVM-5a10s27.pem'}) => {"failed": true, "item": {"dest": "/etc/contrail/ssl/certs/server.pem", "src": "/etc/contrail_smgr/puppet/ssl/ContrailVM-5a10s27.pem"}, "msg": "Unable to find '/etc/contrail_smgr/puppet/ssl/ContrailVM-5a10s27.pem' in expected paths."}"
"2017-08-18 17:28:07,542-INFO-sm_ansible_callback.py:43-append(): fatal: [10.87.36.19]: FAILED! => (item - None) {"changed": false, "failed": true, "msg": "One or more items failed"}"
"2017-08-18 17:28:07,545-INFO-sm_ansible_callback.py:43-append(): fatal: [10.87.36.19] (item={u'dest': u'/etc/contrail/ssl/private/server-privkey.pem', u'src': u'/etc/contrail_smgr/puppet/ssl/ContrailVM-5a10s27-privkey.pem'}) => {"failed": true, "item": {"dest": "/etc/contrail/ssl/private/server-privkey.pem", "src": "/etc/contrail_smgr/puppet/ssl/ContrailVM-5a10s27-privkey.pem"}, "msg": "Unable to find '/etc/contrail_smgr/puppet/ssl/ContrailVM-5a10s27-privkey.pem' in expected paths."}"
"2017-08-18 17:28:07,548-INFO-sm_ansible_callback.py:43-append(): RUNNING HANDLER [openstack/compute : restart nova-compute]"
"2017-08-18 17:28:07,551-INFO-sm_ansible_callback.py:43-append(): RUNNING HANDLER [openstack/compute : restart libvirt]"
"2017-08-18 17:28:07,555-INFO-sm_ansible_utils.py:400-send_REST_request(): Sending post request to http://10.87.36.15:9002/ansible_status?server_id=10.87.36.10&state=provision_completed"
"2017-08-18 17:28:07,558-DEBUG-server_mgr_status.py:132-put_ansible_status(): Server status Data 5a10s31 provision_completed 2017_08_18__17_28_07"
"2017-08-18 17:28:07,662-DEBUG-server_mgr_status.py:135-put_ansible_status(): ######### cluster is cluster-vcenter-compute ##########"
"2017-08-18 17:28:07,665-INFO-sm_ansible_callback.py:43-append(): PLAY RECAP [10.87.36.10] : ok: 83 changed: 27 unreachable: 0 skipped: 179 failed: 0"
"2017-08-18 17:28:07,667-INFO-sm_ansible_utils.py:400-send_REST_request(): Sending post request to http://10.87.36.15:9002/ansible_status?server_id=10.87.36.11&state=provision_completed"
"2017-08-18 17:28:07,669-DEBUG-server_mgr_status.py:132-put_ansible_status(): Server status Data 5a10s30 provision_completed 2017_08_18__17_28_07"
"2017-08-18 17:28:07,779-DEBUG-server_mgr_status.py:135-put_ansible_status(): ######### cluster is cluster-vcenter-compute ##########"
"2017-08-18 17:28:07,782-INFO-sm_ansible_callback.py:43-append(): PLAY RECAP [10.87.36.11] : ok: 83 changed: 27 unreachable: 0 skipped: 179 failed: 0"
"2017-08-18 17:28:07,784-INFO-sm_ansible_utils.py:400-send_REST_request(): Sending post request to http://10.87.36.15:9002/ansible_status?server_id=10.87.36.12&state=provision_completed"
"2017-08-18 17:28:07,787-DEBUG-server_mgr_status.py:132-put_ansible_status(): Server status Data 5a10s29 provision_completed 2017_08_18__17_28_07"
"2017-08-18 17:28:07,888-DEBUG-server_mgr_status.py:135-put_ansible_status(): ######### cluster is cluster-vcenter-compute ##########"
"2017-08-18 17:28:07,891-INFO-sm_ansible_callback.py:43-append(): PLAY RECAP [10.87.36.12] : ok: 91 changed: 30 unreachable: 0 skipped: 174 failed: 0"
"2017-08-18 17:28:07,893-INFO-sm_ansible_utils.py:400-send_REST_request(): Sending post request to http://10.87.36.15:9002/ansible_status?server_id=10.87.36.19&state=provision_failed"
"2017-08-18 17:28:07,896-DEBUG-server_mgr_status.py:132-put_ansible_status(): Server status Data 5a10s27 provision_failed 2017_08_18__17_28_07"
"2017-08-18 17:28:07,989-DEBUG-server_mgr_status.py:135-put_ansible_status(): ######### cluster is cluster-vcenter-compute ##########"
"2017-08-18 17:28:07,992-INFO-sm_ansible_callback.py:43-append(): PLAY RECAP [10.87.36.19] : ok: 81 changed: 20 unreachable: 0 skipped: 244 failed: 1"
"2017-08-18 17:28:07,994-INFO-sm_ansible_utils.py:400-send_REST_request(): Sending post request to http://10.87.36.15:9002/ansible_status?server_id=localhost&state=provision_completed"
"2017-08-18 17:28:07,996-DEBUG-server_mgr_status.py:111-put_ansible_status(): Server status Data provision_completed localhost 2017_08_18__17_28_07"
"2017-08-18 17:28:07,999-INFO-sm_ansible_callback.py:43-append(): PLAY RECAP [localhost] : ok: 52 changed: 18 unreachable: 0 skipped: 7 failed: 0"
"2017-08-18 17:28:08,001-INFO-sm_ansible_utils.py:400-send_REST_request(): Sending post request to http://10.87.36.15:9002/ansible_status?server_id=10.87.36.10&state=provision_failed"
"2017-08-18 17:28:08,004-DEBUG-server_mgr_status.py:132-put_ansible_status(): Server status Data 5a10s31 provision_failed 2017_08_18__17_28_08"
"2017-08-18 17:28:08,123-DEBUG-server_mgr_status.py:135-put_ansible_status(): ######### cluster is cluster-vcenter-compute ##########"
"2017-08-18 17:28:08,126-INFO-sm_ansible_utils.py:400-send_REST_request(): Sending post request to http://10.87.36.15:9002/ansible_status?server_id=10.87.36.11&state=provision_failed"
"2017-08-18 17:28:08,129-DEBUG-server_mgr_status.py:132-put_ansible_status(): Server status Data 5a10s30 provision_failed 2017_08_18__17_28_08"
"2017-08-18 17:28:08,240-DEBUG-server_mgr_status.py:135-put_ansible_status(): ######### cluster is cluster-vcenter-compute ##########"
"2017-08-18 17:28:08,244-INFO-sm_ansible_utils.py:400-send_REST_request(): Sending post request to http://10.87.36.15:9002/ansible_status?server_id=10.87.36.12&state=provision_failed"
"2017-08-18 17:28:08,246-DEBUG-server_mgr_status.py:132-put_ansible_status(): Server status Data 5a10s29 provision_failed 2017_08_18__17_28_08"
"2017-08-18 17:28:08,358-DEBUG-server_mgr_status.py:135-put_ansible_status(): ######### cluster is cluster-vcenter-compute ##########"
"2017-08-18 17:28:08,361-INFO-sm_ansible_utils.py:400-send_REST_request(): Sending post request to http://10.87.36.15:9002/ansible_status?server_id=10.87.36.19&state=provision_failed"
"2017-08-18 17:28:08,364-DEBUG-server_mgr_status.py:132-put_ansible_status(): Server status Data 5a10s27 provision_failed 2017_08_18__17_28_08"
"2017-08-18 17:28:08,458-DEBUG-server_mgr_status.py:135-put_ansible_status(): ######### cluster is cluster-vcenter-compute ##########"
"2017-08-18 17:28:08,462-ERROR-sm_ansible_playbook.py:751-run_playbook(): Playbook Failed: /opt/contrail/server_manager/ansible/playbooks/contrail_4_0_1_34_vc_new/playbooks/site.yml"
"2017-08-18 17:28:08,482-INFO-sm_ansible_server.py:43-run(): Process Done"

[Sarath (nsarath)]

nsarath@ubuntu-build04:/auto/cores/1711772$ ls -l *
-rwxrwxrwx 1 nsarath test 138782720 Aug 18 18:21 SM-log.tar

[kamlesh parmar (kparmar)]

Upgrade to latest CB, some code change have gone in since B34. Use the new json files i provided. And add openstack role to your cluster servers. Please re-test and open new bug as you see issues.

[Sarath (nsarath)]

As initial build given to QA is #34 and as the basis of closure is some code changes went in recent days, so marking the bug state accordingly.