upgrade_contrail from R2.10 to R2.11 aborts

[wenqing liang]

Upgrade_contrail from r2.10-39 to r2.11-53 failed:

2015-04-16 12:52:48:279945:
2015-04-16 12:52:10:464342: Warning: sudo() received nonzero return code 1 while executing 'service supervisor-support-service stop'!
2015-04-16 12:52:10:464342:
2015-04-16 12:52:10:464342: Traceback (most recent call last):
2015-04-16 12:52:48:280930: File "/usr/local/lib/python2.7/dist-packages/fabric/main.py", line 743, in main
2015-04-16 12:52:48:280989: *args, **kwargs
2015-04-16 12:52:48:281229: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 368, in execute
2015-04-16 12:52:48:281282: multiprocessing
2015-04-16 12:52:48:281452: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 264, in _execute
2015-04-16 12:52:48:281500: return task.run(*args, **kwargs)
2015-04-16 12:52:48:281641: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
2015-04-16 12:52:48:281689: return self.wrapped(*args, **kwargs)
2015-04-16 12:52:48:281810: File "/usr/local/lib/python2.7/dist-packages/fabric/decorators.py", line 53, in inner_decorator
2015-04-16 12:52:48:281858: return func(*args, **kwargs)
2015-04-16 12:52:48:281963: File "/opt/contrail/utils/fabfile/tasks/upgrade.py", line 961, in upgrade_contrail
2015-04-16 12:52:48:282012: execute('upgrade_orchestrator', from_rel, pkg)
2015-04-16 12:52:48:282380: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 368, in execute
2015-04-16 12:52:48:282430: multiprocessing
2015-04-16 12:52:48:282597: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 264, in _execute
2015-04-16 12:52:48:282644: return task.run(*args, **kwargs)
2015-04-16 12:52:48:282787: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
2015-04-16 12:52:48:282834: return self.wrapped(*args, **kwargs)
2015-04-16 12:52:48:282957: File "/usr/local/lib/python2.7/dist-packages/fabric/decorators.py", line 53, in inner_decorator
2015-04-16 12:52:48:283006: return func(*args, **kwargs)
2015-04-16 12:52:48:283106: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 168, in __call__
2015-04-16 12:52:48:283153: return self.run(*args, **kwargs)
2015-04-16 12:52:48:283276: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
2015-04-16 12:52:48:283323: return self.wrapped(*args, **kwargs)
2015-04-16 12:52:48:283446: File "/opt/contrail/utils/fabfile/tasks/upgrade.py", line 943, in upgrade_orchestrator
2015-04-16 12:52:48:283493: execute('upgrade_openstack', from_rel, pkg)
2015-04-16 12:52:48:283827: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 368, in execute
2015-04-16 12:52:48:283877: multiprocessing
2015-04-16 12:52:48:284043: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 264, in _execute
2015-04-16 12:52:48:284090: return task.run(*args, **kwargs)
2015-04-16 12:52:48:284233: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
2015-04-16 12:52:48:284283: return self.wrapped(*args, **kwargs)
2015-04-16 12:52:48:284406: File "/usr/local/lib/python2.7/dist-packages/fabric/decorators.py", line 53, in inner_decorator
2015-04-16 12:52:48:284453: return func(*args, **kwargs)
2015-04-16 12:52:48:284553: File "/opt/contrail/utils/fabfile/tasks/upgrade.py", line 569, in upgrade_openstack
2015-04-16 12:52:48:284600: execute("upgrade_openstack_node", from_rel, pkg, env.host_string)
2015-04-16 12:52:48:284834: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 405, in execute
2015-04-16 12:52:48:284884: results['<local-only>'] = task.run(*args, **new_kwargs)
2015-04-16 12:52:48:285061: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
2015-04-16 12:52:48:285110: return self.wrapped(*args, **kwargs)
2015-04-16 12:52:48:285262: File "/opt/contrail/utils/fabfile/tasks/upgrade.py", line 581, in upgrade_openstack_node
2015-04-16 12:52:48:285312: execute('backup_config_node', from_rel, host_string)
2015-04-16 12:52:48:285551: File "/usr/local
2015-04-16 12:52:48:280321: Disconnecting from 10.87.129.174... done.
2015-04-16 12:52:48:289875: Disconnecting from 10.87.129.175... done.
2015-04-16 12:52:48:385670: Disconnecting from 10.87.129.173... done.
2015-04-16 12:52:48:453037: /lib/python2.7/dist-packages/fabric/tasks.py", line 405, in execute
2015-04-16 12:52:48:285601: results['<local-only>'] = task.run(*args, **new_kwargs)
2015-04-16 12:52:48:285799: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
2015-04-16 12:52:48:285850: return self.wrapped(*args, **kwargs)
2015-04-16 12:52:48:285974: File "/opt/contrail/utils/fabfile/tasks/upgrade.py", line 428, in backup_config_node
2015-04-16 12:52:48:286024: if host_string in env.roledefs['tsn'] or host_string in env.roledefs['toragent']:
2015-04-16 12:52:48:286223: KeyError: 'tsn'
2015-04-16 12:52:48:286378: root@cmbu-vse2100-1:/opt/contrail/utils#

2015-04-16 12:52:48:286378: root@cmbu-vse2100-1:/opt/contrail/utils# contrail-version
Package Version Build-ID | Repo | Package Name
-------------------------------------- ------------------------------ ----------------------------------
contrail-analytics 2.10-39 53
contrail-config 2.10-39 53
contrail-config-openstack 2.10-39 53
contrail-control 2.10-39 53
contrail-dns 2.10-39 53
contrail-fabric-utils 2.11-53 53
contrail-heat 2.10-39 53
contrail-install-packages 2.11-53~icehouse 53
contrail-lib 2.10-39 53
contrail-nodemgr 2.10-39 53
contrail-openstack 2.10-39 53
contrail-openstack-analytics 2.10-39 53
contrail-openstack-config 2.10-39 53
contrail-openstack-control 2.10-39 53
contrail-openstack-dashboard 2.10-39 53
contrail-openstack-database 2.10-39 53
contrail-openstack-webui 2.10-39 53
contrail-setup 2.11-53 53
contrail-utils 2.10-39 53
contrail-web-controller 2.10-39 53
contrail-web-core 2.10-39 53
ifmap-python-client 0.1-2 53
ifmap-server 0.3.2-1contrail1 53
neutron-plugin-contrail 2.10-39 53
nova-api 1:2014.1.3-0ubuntu1~cloud0.2contrail53
nova-common 1:2014.1.3-0ubuntu1~cloud0.2contrail53
nova-conductor 1:2014.1.3-0ubuntu1~cloud0.2contrail53
nova-console 1:2014.1.3-0ubuntu1~cloud0.2contrail53
nova-consoleauth 1:2014.1.3-0ubuntu1~cloud0.2contrail53
nova-novncproxy 1:2014.1.3-0ubuntu1~cloud0.2contrail53
nova-objectstore 1:2014.1.3-0ubuntu1~cloud0.2contrail53
nova-scheduler 1:2014.1.3-0ubuntu1~cloud0.2contrail53
python-contrail 2.10-39 53
python-neutronclient 2:2.3.4-0ubuntu1.2contrail 53
python-nova 1:2014.1.3-0ubuntu1~cloud0.2contrail53
root@cmbu-vse2100-1:/opt/contrail/utils#

[Shweta Naik (stnaik)]

I have committed the fix.
https://review.opencontrail.org/#/c/9208/1

[Shweta Naik (stnaik)]

https://review.opencontrail.org/#/c/9208/2

[wenqing liang (wliang)]

Pending confirmation from Ignatious on fix merge.