Activity log for bug #1690991

Date Who What changed Old value New value Message
2017-05-16 04:56:30 Vivek krishnappa bug added bug
2017-05-16 05:10:30 Sudheendra Rao juniperopenstack: importance Undecided Critical
2017-05-16 05:10:42 Sudheendra Rao juniperopenstack: assignee Dheeraj Gautam (dgautam)
2017-05-16 05:10:48 Sudheendra Rao juniperopenstack: milestone r4.0.0.0-fcs
2017-05-16 05:10:50 Sudheendra Rao summary R4.0:4:: Installation of SM and openstack on the same node fails R4.0:4:: openstack provision fails on 16.04.2 on single node SMLite setup.
2017-05-16 05:11:48 Sudheendra Rao description Provisioning of server manager and openstack fails on a single node setup for ubunut:16-04, & build R4.0:4 (newton). The issue persists even after applying the patch listed in "https://review.opencontrail.org/#/c/31519/1/contrail_provisioning/compute/scripts/compute-server-setup.sh" and changing “auth_type=v3password” to password in file file /etc/nova/nova.conf. Sys logs points to “ERROR oslo_service.service err.raise_mysql_exception(self._data) May 15 19:44:04 nodea10 nova-conductor[2497]: 2017-05-15 19:44:04.603 18860 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception. Logs:: ===================== root@nodea10:~# server-manager status server +---------+---------------------+---------------+-------------------+ | id | status | ip_address | mac_address | +---------+---------------------+---------------+-------------------+ | nodea10 | openstack_completed | 10.204.216.48 | 00:25:90:A5:3B:1A | +---------+---------------------+---------------+-------------------+ root@nodea10:~# root@nodea10:~# server-manager display cluster -d { "cluster": [ { "base_image_id": "", "email": "", "id": "cluster1", "package_image_id": "", "parameters": { "domain": "englab.juniper.net", "gateway": "10.204.216.254", "provision": { "contrail": { "config": { "config_ip_list": [ "10.204.216.48" ] }, "storage": { "last_osd_num": 0, "osd_bootstrap_key": "AQA/WhlZOkZ6FRAAgauhod7N7wx0+iTSYWi74g==", "pool_keys": { "images": "AQC7WhlZ7DF7DBAAk576CTksbdda3YEZuHg6fQ==", "volumes": "AQC7WhlZ3RqhCxAANF02IiMvfzZAy53vJegfhg==" }, "pool_secret": { "images": "68b8330c-82bc-41cd-a540-5e640820be4d", "volumes": "6ab41568-56cd-4421-bb3c-11f7e2b6b339" }, "storage_admin_key": "AQA/WhlZ0uISFhAAlchCo73mvXc581uYQPEFXg==", "storage_enabled": 0, "storage_fsid": "3b99c6f7-2abc-4de5-9b06-934c9f9bb58f", "storage_monitor_secret": "AQA/WhlZhszhFBAADthB5iU9EBMaSN+/YTanuQ==", "storage_virsh_uuid": "3bd83fde-500d-400f-ac31-36774cd5d830" } }, "contrail_4": { "contrail_analytics_log_volumes": [ "/var/log/contrail/analytics:/var/log/contrail" ], "contrail_analyticsdb_log_volumes": [ "/var/log/contrail/analyticsdb:/var/log/contrail" ], "contrail_controller_log_volumes": [ "/var/log/contrail/controller:/var/log/contrail", "/var/log/zookeeper:/var/log/zookeeper" ], "controller_config": { "bgp_asn": "64512", "encap_priority": "VXLAN,MPLSoUDP,MPLSoGRE", "external_routers_list": { "blr-xm1": "10.204.216.253" } } }, "openstack": { "ceilometer": { "mongo": "*****", "password": "*****" }, "cinder": { "password": "*****" }, "glance": { "password": "*****" }, "heat": { "encryption_key": "*****", "password": "*****" }, "horizon": { "password": "*****" }, "keystone": { "admin_password": "*****", "admin_token": "*****" }, "mysql": { "root_password": "*****", "service_password": "*****" }, "neutron": { "password": "*****" }, "nova": { "password": "*****" }, "openstack_manage_amqp": true, "swift": { "password": "*****" } } }, "storage_fsid": "38f1b385-ae6d-4bb3-9e44-11c9a1988276", "storage_virsh_uuid": "c3eaa0a7-e512-448d-a80f-f910994bb14c", "subnet_mask": "255.255.255.0", "uuid": "50ff2e3d-9d0d-48dd-acc8-0b168082b3b6" }, "provision_role_sequence": "{'completed': [(u'nodea10', 'openstack', '2017_05_16__04_20_36')], 'steps': [[(u'nodea10', 'post_provision')]]}", "provisioned_id": null } ] } ==================================================== Dev_comments:: ===================== On newton, keystone is managed by apache, during provisioning of keystone we inadvertently removed all the other services managed by apache. Info: Computing checksum on file /etc/apache2/sites-available/puppetmaster.conf Info: /Stage[openstack]/Apache/File[/etc/apache2/sites-available/puppetmaster.conf]: Filebucketed /etc/apache2/sites-available/puppetmaster.conf to puppet with sum f32918b82860016169c6dad8c10513a4 Notice: /Stage[openstack]/Apache/File[/etc/apache2/sites-available/puppetmaster.conf]/ensure: removed Info: Computing checksum on file /etc/apache2/sites-available/smgr.conf Info: /Stage[openstack]/Apache/File[/etc/apache2/sites-available/smgr.conf]: Filebucketed /etc/apache2/sites-available/smgr.conf to puppet with sum 7c58aeb7b808757479392389e7267145 Notice: /Stage[openstack]/Apache/File[/etc/apache2/sites-available/smgr.conf]/ensure: removed Info: /etc/apache2/sites-available: Scheduling refresh of Class[Apache::Service] Notice: /Stage[openstack]/Apache/File[/etc/apache2/sites-enabled/puppetmaster.conf]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/sites-enabled/smgr.conf]/ensure: removed Info: /etc/apache2/sites-enabled: Scheduling refresh of Class[Apache::Service] Notice: executed verify-rabbitmq Notice: /Stage[openstack]/Contrail::Verify_rabbitmq/Notify[executed verify-rabbitmq]/message: defined 'message' as 'executed verify-rabbitmq' Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/headers.load]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/mpm_worker.conf]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/mpm_worker.load]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/passenger.conf]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/passenger.load]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/socache_shmcb.load]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/ssl.conf]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/ssl.load]/ensure: removed Info: /etc/apache2/mods-enabled: Scheduling refresh of Class[Apache::Service] Provisioning of openstack fails on a single node SMLite setup for ubunut:16-04, & build R4.0:4 (newton). The issue persists even after applying the patch listed in "https://review.opencontrail.org/#/c/31519/1/contrail_provisioning/compute/scripts/compute-server-setup.sh" and changing “auth_type=v3password” to password in file file /etc/nova/nova.conf. Sys logs points to “ERROR oslo_service.service err.raise_mysql_exception(self._data) May 15 19:44:04 nodea10 nova-conductor[2497]: 2017-05-15 19:44:04.603 18860 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception. Logs:: ===================== root@nodea10:~# server-manager status server +---------+---------------------+---------------+-------------------+ | id | status | ip_address | mac_address | +---------+---------------------+---------------+-------------------+ | nodea10 | openstack_completed | 10.204.216.48 | 00:25:90:A5:3B:1A | +---------+---------------------+---------------+-------------------+ root@nodea10:~# root@nodea10:~# server-manager display cluster -d {     "cluster": [         {             "base_image_id": "",             "email": "",             "id": "cluster1",             "package_image_id": "",             "parameters": {                 "domain": "englab.juniper.net",                 "gateway": "10.204.216.254",                 "provision": {                     "contrail": {                         "config": {                             "config_ip_list": [                                 "10.204.216.48"                             ]                         },                         "storage": {                             "last_osd_num": 0,                             "osd_bootstrap_key": "AQA/WhlZOkZ6FRAAgauhod7N7wx0+iTSYWi74g==",                             "pool_keys": {                                 "images": "AQC7WhlZ7DF7DBAAk576CTksbdda3YEZuHg6fQ==",                                 "volumes": "AQC7WhlZ3RqhCxAANF02IiMvfzZAy53vJegfhg=="                             },                             "pool_secret": {                                 "images": "68b8330c-82bc-41cd-a540-5e640820be4d",                                 "volumes": "6ab41568-56cd-4421-bb3c-11f7e2b6b339"                             },                             "storage_admin_key": "AQA/WhlZ0uISFhAAlchCo73mvXc581uYQPEFXg==",                             "storage_enabled": 0,                             "storage_fsid": "3b99c6f7-2abc-4de5-9b06-934c9f9bb58f",                             "storage_monitor_secret": "AQA/WhlZhszhFBAADthB5iU9EBMaSN+/YTanuQ==",                             "storage_virsh_uuid": "3bd83fde-500d-400f-ac31-36774cd5d830"                         }                     },                     "contrail_4": {                         "contrail_analytics_log_volumes": [                             "/var/log/contrail/analytics:/var/log/contrail"                         ],                         "contrail_analyticsdb_log_volumes": [                             "/var/log/contrail/analyticsdb:/var/log/contrail"                         ],                         "contrail_controller_log_volumes": [                             "/var/log/contrail/controller:/var/log/contrail",                             "/var/log/zookeeper:/var/log/zookeeper"                         ],                         "controller_config": {                             "bgp_asn": "64512",                             "encap_priority": "VXLAN,MPLSoUDP,MPLSoGRE",                             "external_routers_list": {                                 "blr-xm1": "10.204.216.253"                             }                         }                     },                     "openstack": {                         "ceilometer": {                             "mongo": "*****",                             "password": "*****"                         },                         "cinder": {                             "password": "*****"                         },                         "glance": {                             "password": "*****"                         },                         "heat": {                             "encryption_key": "*****",                             "password": "*****"                         },                         "horizon": {                             "password": "*****"                         },                         "keystone": {                             "admin_password": "*****",                             "admin_token": "*****"                         },                         "mysql": {                             "root_password": "*****",                             "service_password": "*****"                         },                         "neutron": {                             "password": "*****"                         },                         "nova": {                             "password": "*****"                         },                         "openstack_manage_amqp": true,                         "swift": {                             "password": "*****"                         }                     }                 },                 "storage_fsid": "38f1b385-ae6d-4bb3-9e44-11c9a1988276",                 "storage_virsh_uuid": "c3eaa0a7-e512-448d-a80f-f910994bb14c",                 "subnet_mask": "255.255.255.0",                 "uuid": "50ff2e3d-9d0d-48dd-acc8-0b168082b3b6"             },             "provision_role_sequence": "{'completed': [(u'nodea10', 'openstack', '2017_05_16__04_20_36')], 'steps': [[(u'nodea10', 'post_provision')]]}",             "provisioned_id": null         }     ] } ==================================================== Dev_comments:: ===================== On newton, keystone is managed by apache, during provisioning of keystone we inadvertently removed all the other services managed by apache. Info: Computing checksum on file /etc/apache2/sites-available/puppetmaster.conf Info: /Stage[openstack]/Apache/File[/etc/apache2/sites-available/puppetmaster.conf]: Filebucketed /etc/apache2/sites-available/puppetmaster.conf to puppet with sum f32918b82860016169c6dad8c10513a4 Notice: /Stage[openstack]/Apache/File[/etc/apache2/sites-available/puppetmaster.conf]/ensure: removed Info: Computing checksum on file /etc/apache2/sites-available/smgr.conf Info: /Stage[openstack]/Apache/File[/etc/apache2/sites-available/smgr.conf]: Filebucketed /etc/apache2/sites-available/smgr.conf to puppet with sum 7c58aeb7b808757479392389e7267145 Notice: /Stage[openstack]/Apache/File[/etc/apache2/sites-available/smgr.conf]/ensure: removed Info: /etc/apache2/sites-available: Scheduling refresh of Class[Apache::Service] Notice: /Stage[openstack]/Apache/File[/etc/apache2/sites-enabled/puppetmaster.conf]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/sites-enabled/smgr.conf]/ensure: removed Info: /etc/apache2/sites-enabled: Scheduling refresh of Class[Apache::Service] Notice: executed verify-rabbitmq Notice: /Stage[openstack]/Contrail::Verify_rabbitmq/Notify[executed verify-rabbitmq]/message: defined 'message' as 'executed verify-rabbitmq' Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/headers.load]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/mpm_worker.conf]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/mpm_worker.load]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/passenger.conf]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/passenger.load]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/socache_shmcb.load]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/ssl.conf]/ensure: removed Notice: /Stage[openstack]/Apache/File[/etc/apache2/mods-enabled/ssl.load]/ensure: removed Info: /etc/apache2/mods-enabled: Scheduling refresh of Class[Apache::Service]
2017-05-16 05:11:54 Sudheendra Rao information type Proprietary Public
2017-05-16 05:12:03 Sudheendra Rao tags provisioning provisioning sanity server-manager
2017-05-16 16:41:34 Jeba Paulaiyan nominated for series juniperopenstack/trunk
2017-05-16 16:41:34 Jeba Paulaiyan bug task added juniperopenstack/trunk
2017-05-16 16:41:34 Jeba Paulaiyan nominated for series juniperopenstack/r4.0
2017-05-16 16:41:34 Jeba Paulaiyan bug task added juniperopenstack/r4.0
2017-05-16 16:41:40 Jeba Paulaiyan juniperopenstack/r4.0: importance Undecided Critical
2017-05-16 16:41:46 Jeba Paulaiyan juniperopenstack/r4.0: assignee Dheeraj Gautam (dgautam)
2017-05-16 16:41:50 Jeba Paulaiyan juniperopenstack/r4.0: milestone r4.0.0.0-fcs
2017-05-16 16:41:53 Jeba Paulaiyan juniperopenstack/trunk: milestone r4.0.0.0-fcs r4.1.0.0-fcs
2017-05-20 05:06:17 Jeba Paulaiyan tags provisioning sanity server-manager blocker provisioning sanity server-manager
2017-05-20 18:54:26 OpenContrail Admin juniperopenstack/r4.0: status New In Progress
2017-05-20 18:54:29 OpenContrail Admin juniperopenstack/trunk: status New In Progress
2017-05-25 23:39:09 Abhay Joshi juniperopenstack/r4.0: status In Progress Fix Committed
2017-05-25 23:39:14 Abhay Joshi juniperopenstack/trunk: status In Progress Fix Committed