Comment 0 for bug 1613159

SM mitaka HA R3.1 build 25 provision gets stuck at config_started due to keystone conflict among Openstack nodes

setup
SM : nodej8

Targets : nodeg21,nodeg33,nodec58,nodeg17,nodec38

root@nodej8:~# server-manager show server --select id,roles,cluster_id,ip_address
{
    "server": [
        {
            "cluster_id": "cluster5sanity",
            "id": "nodeg21",
            "ip_address": "10.204.217.61",
            "roles": [
                "control",
                "webui",
                "openstack",
                "database",
                "collector",
                "config"
            ]
        },
        {
            "cluster_id": "cluster5sanity",
            "id": "nodec38",
            "ip_address": "10.204.217.23",
            "roles": [
                "compute"
            ]
        },
        {
            "cluster_id": "cluster5sanity",
            "id": "nodeg17",
            "ip_address": "10.204.217.57",
            "roles": [
                "compute"
            ]
        },
        {
            "cluster_id": "cluster5sanity",
            "id": "nodec58",
            "ip_address": "10.204.217.98",
            "roles": [
                "control",
                "webui",
                "openstack",
                "database",
                "collector",
                "config"
            ]
        },
        {
            "cluster_id": "cluster5sanity",
            "id": "nodeg33",
            "ip_address": "10.204.217.73",
            "roles": [
                "control",
                "webui",
                "openstack",
                "database",
                "collector",
                "config"
            ]
        }
    ]
}
root@nodej8:~#

cfgm0 node nodeg21 syslog

Aug 14 11:20:29 nodeg21 puppet-agent[28239]: contrail contrail_exec_provision_control is python exec_provision_control.py --api_server_ip "10.204.217.176" --api_server_port 8082 --host_name_list "nodeg21,nodec58,nodeg33" --host_ip_list "10.204.217.61,10.204.217.98,10.204.217.73" --router_asn "64512" --mt_options "admin,contrail123,admin" && echo exec-provision-control >> /etc/contrail/contrail_config_exec.out
Aug 14 11:20:29 nodeg21 puppet-agent[28239]: (/Stage[config]/Contrail::Exec_provision_control/Notify[contrail contrail_exec_provision_control is python exec_provision_control.py --api_server_ip "10.204.217.176" --api_server_port 8082 --host_name_list "nodeg21,nodec58,nodeg33" --host_ip_list "10.204.217.61,10.204.217.98,10.204.217.73" --router_asn "64512" --mt_options "admin,contrail123,admin" && echo exec-provision-control >> /etc/contrail/contrail_config_exec.out]/message) defined 'message' as 'contrail contrail_exec_provision_control is python exec_provision_control.py --api_server_ip "10.204.217.176" --api_server_port 8082 --host_name_list "nodeg21,nodec58,nodeg33" --host_ip_list "10.204.217.61,10.204.217.98,10.204.217.73" --router_asn "64512" --mt_options "admin,contrail123,admin" && echo exec-provision-control >> /etc/contrail/contrail_config_exec.out'
Aug 14 11:22:16 nodeg21 kernel: [ 1835.798551] init: supervisor-config main process (6341) killed by TERM signal
Aug 14 11:22:28 nodeg21 puppet-agent[28239]: python exec_provision_control.py --api_server_ip "10.204.217.176" --api_server_port 8082 --host_name_list "nodeg21,nodec58,nodeg33" --host_ip_list "10.204.217.61,10.204.217.98,10.204.217.73" --router_asn "64512" --mt_options "admin,contrail123,admin" && echo exec-provision-control >> /etc/contrail/contrail_config_exec.out returned 1 instead of one of [0]
Aug 14 11:22:28 nodeg21 puppet-agent[28239]: (/Stage[config]/Contrail::Exec_provision_control/Exec[exec-provision-control]/returns) change from notrun to 0 failed: python exec_provision_control.py --api_server_ip "10.204.217.176" --api_server_port 8082 --host_name_list "nodeg21,nodec58,nodeg33" --host_ip_list "10.204.217.61,10.204.217.98,10.204.217.73" --router_asn "64512" --mt_options "admin,contrail123,admin" && echo exec-provision-control >> /etc/contrail/contrail_config_exec.out returned 1 instead of one of [0]

Issue is due to the parameter default_domain_id /etc/keystone/keystone.conf is not same among the openstack nodes

WorkAround :

1) grep default_domain_id /etc/keystone/keystone.conf among the openstack nodes. This might be of 2) make sure all the openstack nodes have same default_domain_id ;
3) Restart the keystone service on all the openstack nodes
3) observe that the provision gets completed succesfully