smlite provision fails due to old json format conversion of testbed.py

[wenqing liang]

r3.1.0.0-19 mitaka.

provision stuck at u'webui' on dependency issues that came because smlite converted testbed.py into unsupported old json formats.

Aug 10 10:47:44 cmbu-ceph-perf1 puppet-agent[11185]: (/Stage[webui]/Contrail::Webui/Contrail::Lib::Report_status[webui_completed]/Exec[contrail-status-webui_completed]) Dependency Package[contrail-web-storage] has failures: true
Aug 10 10:47:44 cmbu-ceph-perf1 puppet-agent[11185]: (/Stage[webui]/Contrail::Webui/Contrail::Lib::Report_status[webui_completed]/Exec[contrail-status-webui_completed]) Skipping because of failed dependencies

root@cmbu-ceph-perf1:/var/log/contrail/sm_provisioning/2016_08_10__08_21_49# ls
cluster.json image.json preconfig.log provision_2016_08_10__08_21_49.log server.json test_parser.log
root@cmbu-ceph-perf1:/var/log/contrail/sm_provisioning/2016_08_10__08_21_49# cat server.json
{
    "server": [
        {
            "cluster_id": "wqlsmlitecluster",
            "contrail": {
                "control_data_interface": "p6p2"
            },
            "domain": "englab.juniper.net",
            "id": "cmbu-ceph-perf3",
            "network": {
                "interfaces": [
                    {
                        "default_gateway": "10.87.159.254",
                        "ip_address": "10.87.140.199/19",
                        "mac_address": "00:25:90:92:0e:6c",
                        "name": "eth0"
                    },
                    {
                        "default_gateway": "5.0.0.1",
                        "ip_address": "5.0.0.3/24",
                        "name": "p6p2"
                    }
                ],
                "management_interface": "eth0",
                "provisioning": "kickstart"
            },
            "parameters": {
                "disks": [
                    "/dev/sdc:/dev/sdb",
                    "/dev/sdd:/dev/sdb",
                    "/dev/sde:/dev/sdb",
                    "/dev/sdf:/dev/sdb"
                ],
                "kernel_upgrade": "yes",
                "storage_repo_id": "image_contrail_storage_packages_3_1_0_0_19_mitaka"
            },
            "password": "n1keenA",
            "roles": [
                "storage-compute",
                "compute"
            ]
        },
        {
            "cluster_id": "wqlsmlitecluster",
            "contrail": {
                "control_data_interface": "p6p2"
            },
            "domain": "englab.juniper.net",
            "id": "cmbu-ceph-perf2",
            "network": {
                "interfaces": [
                    {
                        "default_gateway": "10.87.159.254",
                        "ip_address": "10.87.140.198/19",
                        "mac_address": "00:25:90:35:8a:1f",
                        "name": "eth1"
                    },
                    {
                        "default_gateway": "5.0.0.1",
                        "ip_address": "5.0.0.2/24",
                        "name": "p6p2"
                    }
                ],
                "management_interface": "eth1",
                "provisioning": "kickstart"
            },
            "parameters": {
                "disks": [
                    "/dev/sdc:/dev/sdb",
                    "/dev/sdd:/dev/sdb",
                    "/dev/sde:/dev/sdb",
                    "/dev/sdf:/dev/sdb"
                ],
                "kernel_upgrade": "yes",
                "storage_repo_id": "image_contrail_storage_packages_3_1_0_0_19_mitaka"
            },
            "password": "n1keenA",
            "roles": [
                "storage-compute",
                "compute"
            ]
        },
        {
            "cluster_id": "wqlsmlitecluster",
            "contrail": {
                "control_data_interface": "p2p2"
            },
            "domain": "englab.juniper.net",
            "id": "cmbu-ceph-perf1",
            "network": {
                "interfaces": [
                    {
                        "default_gateway": "10.87.159.254",
                        "ip_address": "10.87.140.197/19",
                        "mac_address": "00:25:90:ab:9c:88",
                        "name": "p4p1"
                    },
                    {
                        "default_gateway": "5.0.0.1",
                        "ip_address": "5.0.0.1/24",
                        "name": "p2p2"
                    }
                ],
                "management_interface": "p4p1",
                "provisioning": "kickstart"
            },
            "parameters": {
                "kernel_upgrade": "yes",
                "storage_repo_id": "image_contrail_storage_packages_3_1_0_0_19_mitaka"
            },
            "password": "n1keenA",
            "roles": [
                "control",
                "webui",
                "openstack",
                "collector",
                "database",
                "storage-master",
                "config"
            ]
        },
        {
            "cluster_id": "wqlsmlitecluster",
            "contrail": {
                "control_data_interface": "p6p2"
            },
            "domain": "englab.juniper.net",
            "id": "cmbu-ceph-perf4",
            "network": {
                "interfaces": [
                    {
                        "default_gateway": "10.87.159.254",
                        "ip_address": "10.87.140.200/19",
                        "mac_address": "00:25:90:92:0d:f2",
                        "name": "eth0"
                    },
                    {
                        "default_gateway": "5.0.0.1",
                        "ip_address": "5.0.0.4/24",
                        "name": "p6p2"
                    }
                ],
                "management_interface": "eth0",
                "provisioning": "kickstart"
            },
            "parameters": {
                "disks": [
                    "/dev/sdc:/dev/sdb",
                    "/dev/sdd:/dev/sdb",
                    "/dev/sde:/dev/sdb",
                    "/dev/sdf:/dev/sdb"
                ],
                "kernel_upgrade": "yes",
                "storage_repo_id": "image_contrail_storage_packages_3_1_0_0_19_mitaka"
            },
            "password": "n1keenA",
            "roles": [
                "storage-compute",
                "compute"
            ]
        }
    ]
}
root@cmbu-ceph-perf1:/var/log/contrail/sm_provisioning/2016_08_10__08_21_49# cat cluster.json
{
    "cluster": [
        {
            "id": "wqlsmlitecluster",
            "parameters": {
                "provision": {
                    "contrail": {
                        "control": {
                            "external_bgp": "[('montreal', '10.87.140.185')]",
                            "router_asn": "64512"
                        },
                        "database": {
                            "minimum_diskGB": "256"
                        }
                    },
                    "openstack": {
                        "keystone": {
                            "admin_password": "n1keenA"
                        }
                    }
                }
            }
        }
    ]
}

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/23176
Submitter: Nitish Krishna Kaveri (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/23178
Submitter: Nitish Krishna Kaveri (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/23178
Committed: http://github.org/Juniper/contrail-server-manager/commit/51083e41ef10f92c14b8a501c62862219d7ba9ee
Submitter: Zuul
Branch: master

commit 51083e41ef10f92c14b8a501c62862219d7ba9ee
Author: nitishkrishna <email address hidden>
Date: Wed Aug 10 11:39:27 2016 -0700

Closes-Bug: #1611885 - SM Lite doesn't translate storage server params in new format

The storage_repo_id, storage_osd_disks params are coming in old format

Closes-Bug: #1591453 - SM Lite doesn't correctly pass on osd_bootstrap_key, storage_admin_key and storage_monitor_secret

These params were explicitly being set in cluster params from ini file. They need to be put into testbed.py as below:

storage_monitor_secret = <Ceph format key>
osd_bootstrap_key = <Ceph format key>
storage_admin_key = <Ceph format key>

Also some params for storage which could have been over-ridden by user have been removed from testbed to json conversion:
storage_num_osd
storage_fsid
storage_num_hosts
storage_virsh_uuid
storage_cluster_network

Patch 2:
Added default values for the Ceph keys in contrail-defaults.yaml

Change-Id: Ibc20e5e2428205f8c6199f4ee2610a57ec92d7bb

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/23176
Committed: http://github.org/Juniper/contrail-server-manager/commit/2648f9dd945674f07233cb98e03c4316662bd033
Submitter: Zuul
Branch: R3.1

commit 2648f9dd945674f07233cb98e03c4316662bd033
Author: nitishkrishna <email address hidden>
Date: Wed Aug 10 11:39:27 2016 -0700

Closes-Bug: #1611885 - SM Lite doesn't translate storage server params in new format

The storage_repo_id, storage_osd_disks params are coming in old format

Closes-Bug: #1591453 - SM Lite doesn't correctly pass on osd_bootstrap_key, storage_admin_key and storage_monitor_secret

These params were explicitly being set in cluster params from ini file. They need to be put into testbed.py as below:

storage_monitor_secret = <Ceph format key>
osd_bootstrap_key = <Ceph format key>
storage_admin_key = <Ceph format key>

Also some params for storage which could have been over-ridden by user have been removed from testbed to json conversion:
storage_num_osd
storage_fsid
storage_num_hosts
storage_virsh_uuid
storage_cluster_network

Patch 2:
Added default values for the Ceph keys in contrail-defaults.yaml

Change-Id: Ibc20e5e2428205f8c6199f4ee2610a57ec92d7bb