| 2016-11-14 12:19:52 |
Sudheendra Rao |
description |
contrail-schema fails to start during fab setup_all using contrail-networking package on R3.0.2.x build11.
2016-11-14 16:44:04:689586: [root@10.204.217.172] out: ======= Starting the services ======
2016-11-14 16:44:04:689658: [root@10.204.217.172] out: + for svc in '$msg_svc' '$web_svc' memcached
2016-11-14 16:44:04:689731: [root@10.204.217.172] out: + service rabbitmq-server restart
2016-11-14 16:44:04:689803: [root@10.204.217.172] out: unix:///tmp/supervisord_support_service.sock no such file
2016-11-14 16:44:04:790741: [root@10.204.217.172] out: + for svc in '$msg_svc' '$web_svc' memcached
2016-11-14 16:44:04:790865: [root@10.204.217.172] out: + service apache2 restart
2016-11-14 16:44:04:790944: [root@10.204.217.172] out: apache2: unrecognized service
2016-11-14 16:44:04:791019: [root@10.204.217.172] out: + for svc in '$msg_svc' '$web_svc' memcached
2016-11-14 16:44:04:791093: [root@10.204.217.172] out: + service memcached restart
2016-11-14 16:44:04:791166: [root@10.204.217.172] out: memcached: unrecognized service
2016-11-14 16:44:04:791239: [root@10.204.217.172] out: + service neutron-server restart
2016-11-14 16:44:04:791312: [root@10.204.217.172] out: neutron-server stop/waiting
2016-11-14 16:44:04:807196: [root@10.204.217.172] out: neutron-server start/running, process 19891
2016-11-14 16:44:04:823986: [root@10.204.217.172] out:
2016-11-14 16:44:04:840031:
2016-11-14 16:44:04:844423: [root@10.204.217.172] Executing task 'verify_cfgm'
2016-11-14 16:44:04:844985: [root@10.204.217.172] sudo: service supervisor-config status
2016-11-14 16:44:04:845223: [root@10.204.217.172] out: supervisor-config start/running, process 19190
2016-11-14 16:44:04:911348: [root@10.204.217.172] out:
2016-11-14 16:44:04:911550:
2016-11-14 16:44:04:911848: [root@10.204.217.172] sudo: service contrail-api status
2016-11-14 16:44:04:912070: [root@10.204.217.172] out: contrail-api:0 RUNNING pid 19212, uptime 0:00:24
2016-11-14 16:44:05:053121: [root@10.204.217.172] out:
2016-11-14 16:44:05:060687:
2016-11-14 16:44:05:067685: [root@10.204.217.172] sudo: service contrail-discovery status
2016-11-14 16:44:05:067918: [root@10.204.217.172] out: contrail-discovery:0 RUNNING pid 19393, uptime 0:00:19
2016-11-14 16:44:05:209514: [root@10.204.217.172] out:
2016-11-14 16:44:05:217156:
2016-11-14 16:44:05:223585: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:44:05:223833: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:44:05:373042: [root@10.204.217.172] out:
2016-11-14 16:44:05:374673:
2016-11-14 16:44:05:378899: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:44:25:399503: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:44:25:513619: [root@10.204.217.172] out:
2016-11-14 16:44:25:521738:
2016-11-14 16:44:25:527213: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:44:45:547971: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:44:45:686330: [root@10.204.217.172] out:
2016-11-14 16:44:45:686563:
2016-11-14 16:44:45:686704: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:45:05:704055: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:45:05:825653: [root@10.204.217.172] out:
2016-11-14 16:45:05:825881:
2016-11-14 16:45:05:831655: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:45:25:856442: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:45:25:985384: [root@10.204.217.172] out:
2016-11-14 16:45:25:989995:
2016-11-14 16:45:26:001889: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:45:46:013920: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:45:46:128545: [root@10.204.217.172] out:
2016-11-14 16:45:46:136151:
2016-11-14 16:45:46:143273: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:46:06:152227: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:46:06:366040: [root@10.204.217.172] out:
2016-11-14 16:46:06:366266:
2016-11-14 16:46:06:366604: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:46:26:385599: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:46:26:522646: [root@10.204.217.172] out:
2016-11-14 16:46:26:522837:
2016-11-14 16:46:26:524469: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:46:46:537179: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:46:46:650346: [root@10.204.217.172] out:
2016-11-14 16:46:46:657828:
2016-11-14 16:46:46:664603: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:47:06:677480: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:47:06:813744: [root@10.204.217.172] out:
2016-11-14 16:47:06:813989:
The setup details are:
ontrol Nodes : [u'nodei13-vm2', u'nodei14-vm2']
Compute Nodes : [u'nodea12', u'nodea15', u'nodea17']
Openstack Node : [u'nodek7-vm2']
WebUI Node : nodei13-vm2
Analytics Nodes : [u'nodei13-vm2']
Database Nodes : [u'nodei14-vm2']
Physical Devices : [u'hooper', u"'hooper'"]
The ip-address of the VMs are
nodei13-vm2: 10.204.217.172
nodei14-vm2: 10.204.217.179
nodek7-vm2: 10.204.216.111 |
contrail-schema fails to start during fab setup_all using contrail-networking package on R3.0.2.x build11.
2016-11-14 16:44:04:689586: [root@10.204.217.172] out: ======= Starting the services ======
2016-11-14 16:44:04:689658: [root@10.204.217.172] out: + for svc in '$msg_svc' '$web_svc' memcached
2016-11-14 16:44:04:689731: [root@10.204.217.172] out: + service rabbitmq-server restart
2016-11-14 16:44:04:689803: [root@10.204.217.172] out: unix:///tmp/supervisord_support_service.sock no such file
2016-11-14 16:44:04:790741: [root@10.204.217.172] out: + for svc in '$msg_svc' '$web_svc' memcached
2016-11-14 16:44:04:790865: [root@10.204.217.172] out: + service apache2 restart
2016-11-14 16:44:04:790944: [root@10.204.217.172] out: apache2: unrecognized service
2016-11-14 16:44:04:791019: [root@10.204.217.172] out: + for svc in '$msg_svc' '$web_svc' memcached
2016-11-14 16:44:04:791093: [root@10.204.217.172] out: + service memcached restart
2016-11-14 16:44:04:791166: [root@10.204.217.172] out: memcached: unrecognized service
2016-11-14 16:44:04:791239: [root@10.204.217.172] out: + service neutron-server restart
2016-11-14 16:44:04:791312: [root@10.204.217.172] out: neutron-server stop/waiting
2016-11-14 16:44:04:807196: [root@10.204.217.172] out: neutron-server start/running, process 19891
2016-11-14 16:44:04:823986: [root@10.204.217.172] out:
2016-11-14 16:44:04:840031:
2016-11-14 16:44:04:844423: [root@10.204.217.172] Executing task 'verify_cfgm'
2016-11-14 16:44:04:844985: [root@10.204.217.172] sudo: service supervisor-config status
2016-11-14 16:44:04:845223: [root@10.204.217.172] out: supervisor-config start/running, process 19190
2016-11-14 16:44:04:911348: [root@10.204.217.172] out:
2016-11-14 16:44:04:911550:
2016-11-14 16:44:04:911848: [root@10.204.217.172] sudo: service contrail-api status
2016-11-14 16:44:04:912070: [root@10.204.217.172] out: contrail-api:0 RUNNING pid 19212, uptime 0:00:24
2016-11-14 16:44:05:053121: [root@10.204.217.172] out:
2016-11-14 16:44:05:060687:
2016-11-14 16:44:05:067685: [root@10.204.217.172] sudo: service contrail-discovery status
2016-11-14 16:44:05:067918: [root@10.204.217.172] out: contrail-discovery:0 RUNNING pid 19393, uptime 0:00:19
2016-11-14 16:44:05:209514: [root@10.204.217.172] out:
2016-11-14 16:44:05:217156:
2016-11-14 16:44:05:223585: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:44:05:223833: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:44:05:373042: [root@10.204.217.172] out:
2016-11-14 16:44:05:374673:
2016-11-14 16:44:05:378899: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:44:25:399503: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:44:25:513619: [root@10.204.217.172] out:
2016-11-14 16:44:25:521738:
2016-11-14 16:44:25:527213: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:44:45:547971: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:44:45:686330: [root@10.204.217.172] out:
2016-11-14 16:44:45:686563:
2016-11-14 16:44:45:686704: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:45:05:704055: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:45:05:825653: [root@10.204.217.172] out:
2016-11-14 16:45:05:825881:
2016-11-14 16:45:05:831655: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:45:25:856442: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:45:25:985384: [root@10.204.217.172] out:
2016-11-14 16:45:25:989995:
2016-11-14 16:45:26:001889: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:45:46:013920: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:45:46:128545: [root@10.204.217.172] out:
2016-11-14 16:45:46:136151:
2016-11-14 16:45:46:143273: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:46:06:152227: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:46:06:366040: [root@10.204.217.172] out:
2016-11-14 16:46:06:366266:
2016-11-14 16:46:06:366604: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:46:26:385599: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:46:26:522646: [root@10.204.217.172] out:
2016-11-14 16:46:26:522837:
2016-11-14 16:46:26:524469: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:46:46:537179: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:46:46:650346: [root@10.204.217.172] out:
2016-11-14 16:46:46:657828:
2016-11-14 16:46:46:664603: [root@10.204.217.172] sudo: service contrail-schema status
2016-11-14 16:47:06:677480: [root@10.204.217.172] out: contrail-schema FATAL Exited too quickly (process log may have details)
2016-11-14 16:47:06:813744: [root@10.204.217.172] out:
2016-11-14 16:47:06:813989:
The setup details are:
ontrol Nodes : [u'nodei13-vm2', u'nodei14-vm2']
Compute Nodes : [u'nodea12', u'nodea15', u'nodea17']
Openstack Node : [u'nodek7-vm2']
WebUI Node : nodei13-vm2
Analytics Nodes : [u'nodei13-vm2']
Database Nodes : [u'nodei14-vm2']
Physical Devices : [u'hooper', u"'hooper'"]
The ip-address of the VMs are
nodei13-vm2: 10.204.217.172
nodei14-vm2: 10.204.217.179
nodek7-vm2: 10.204.216.111
Schema log stack-trace:
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.
/usr/bin/contrail-schema in <module>()
6
7 if __name__ == '__main__':
8 sys.exit(
9 load_entry_point('schema-transformer==0.1dev', 'console_scripts', 'contrail-schema')()
10 )
load_entry_point = <function load_entry_point>
/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py in server_main()
842 def server_main():
843 cgitb.enable(format='text')
844 main()
845 # end server_main
846
global main = <function main>
/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py in main(args_str='--conf_file /etc/contrail/contrail-schema.conf -... --conf_file /etc/contrail/contrail-database.conf')
836 _zookeeper_client.master_election(zk_path_pfx + "/schema-transformer",
837 os.getpid(), run_schema_transformer,
838 args)
839 # end main
840
args = Namespace(admin_password='contrail123', admin_te..._ip='10.204.217.179:2181', zk_server_port='2181')
/usr/lib/python2.7/dist-packages/cfgm_common/zkclient.py in master_election(self=<cfgm_common.zkclient.ZookeeperClient object>, path='/schema-transformer', identifier=19768, func=<function run_schema_transformer>, *args=(Namespace(admin_password='contrail123', admin_te..._ip='10.204.217.179:2181', zk_server_port='2181'),), **kwargs={})
335 def master_election(self, path, identifier, func, *args, **kwargs):
336 self._election = self._zk_client.Election(path, identifier)
337 self._election.run(func, *args, **kwargs)
338 # end master_election
339
self = <cfgm_common.zkclient.ZookeeperClient object>
self._election = <kazoo.recipe.election.Election object>
self._election.run = <bound method Election.run of <kazoo.recipe.election.Election object>>
func = <function run_schema_transformer>
args = (Namespace(admin_password='contrail123', admin_te..._ip='10.204.217.179:2181', zk_server_port='2181'),)
kwargs = {}
/usr/lib/python2.7/dist-packages/kazoo/recipe/election.py in run(self=<kazoo.recipe.election.Election object>, func=<function run_schema_transformer>, *args=(Namespace(admin_password='contrail123', admin_te..._ip='10.204.217.179:2181', zk_server_port='2181'),), **kwargs={})
46 try:
47 with self.lock:
48 func(*args, **kwargs)
49
50 except CancelledError:
func = <function run_schema_transformer>
args = (Namespace(admin_password='contrail123', admin_te..._ip='10.204.217.179:2181', zk_server_port='2181'),)
kwargs = {}
/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py in run_schema_transformer(args=Namespace(admin_password='contrail123', admin_te..._ip='10.204.217.179:2181', zk_server_port='2181'))
805 _vnc_lib = VncApi(
806 args.admin_user, args.admin_password, args.admin_tenant_name,
807 args.api_server_ip, args.api_server_port, api_server_use_ssl=args.api_server_use_ssl)
808 connected = True
809 connection_state_update(ConnectionStatus.UP)
args = Namespace(admin_password='contrail123', admin_te..._ip='10.204.217.179:2181', zk_server_port='2181')
args.api_server_ip = '10.204.217.172'
args.api_server_port = '8082'
api_server_use_ssl undefined
args.api_server_use_ssl = 'False'
/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py in __init__(self=<vnc_api.vnc_api.VncApi object>, username='admin', password='contrail123', tenant_name='admin', api_server_host='10.204.217.172', api_server_port='8082', api_server_url=None, conf_file=None, user_info=None, auth_token=None, auth_host=None, auth_port=None, auth_protocol=None, auth_url=None, auth_type=None, wait_for_connect=False, api_server_use_ssl='False', domain_name=None)
320 try:
321 homepage = self._request(rest.OP_GET, self._base_url,
322 retry_on_error=False)
323 self._parse_homepage(homepage)
324 except ServiceUnavailableError as e:
retry_on_error undefined
builtinFalse = False
/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py in _request(self=<vnc_api.vnc_api.VncApi object>, op=2, url='/', data=None, retry_on_error=False, retry_after_authn=False, retry_count=30)
746 self._headers = self._authenticate(content, self._headers)
747 # Recursive call after authentication (max 1 level)
748 content = self._request(op, url, data=data, retry_after_authn=True)
749
750 return content
content = u'Authentication required'
self = <vnc_api.vnc_api.VncApi object>
self._request = <bound method VncApi._request of <vnc_api.vnc_api.VncApi object>>
op = 2
url = '/'
data = None
retry_after_authn = False
builtinTrue = True
/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py in _request(self=<vnc_api.vnc_api.VncApi object>, op=2, url='/', data=None, retry_on_error=True, retry_after_authn=True, retry_count=30)
771 raise BadRequest(status, content)
772 else: # Unknown Error
773 raise HttpError(status, content)
774 # end while True
775
global HttpError = <class 'cfgm_common.exceptions.HttpError'>
status = 401
content = u'Authentication required'
<class 'cfgm_common.exceptions.HttpError'>: HTTP Status: 401 Content: Authentication required
__class__ = <class 'cfgm_common.exceptions.HttpError'>
__delattr__ = <method-wrapper '__delattr__' of HttpError object>
__dict__ = {'content': u'Authentication required', 'status_code': 401}
__doc__ = None
__format__ = <built-in method __format__ of HttpError object>
__getattribute__ = <method-wrapper '__getattribute__' of HttpError object>
__getitem__ = <method-wrapper '__getitem__' of HttpError object>
__getslice__ = <method-wrapper '__getslice__' of HttpError object>
__hash__ = <method-wrapper '__hash__' of HttpError object>
__init__ = <bound method HttpError.__init__ of HttpError()>
__module__ = 'cfgm_common.exceptions'
__new__ = <built-in method __new__ of type object>
__reduce__ = <built-in method __reduce__ of HttpError object>
__reduce_ex__ = <built-in method __reduce_ex__ of HttpError object>
__repr__ = <method-wrapper '__repr__' of HttpError object>
__setattr__ = <method-wrapper '__setattr__' of HttpError object>
__setstate__ = <built-in method __setstate__ of HttpError object>
__sizeof__ = <built-in method __sizeof__ of HttpError object>
__str__ = <bound method HttpError.__str__ of HttpError()>
__subclasshook__ = <built-in method __subclasshook__ of type object>
__unicode__ = <built-in method __unicode__ of HttpError object>
__weakref__ = None
args = ()
content = u'Authentication required'
message = ''
status_code = 401
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/usr/bin/contrail-schema", line 9, in <module>
load_entry_point('schema-transformer==0.1dev', 'console_scripts', 'contrail-schema')()
File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 844, in server_main
main()
File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 838, in main
args)
File "/usr/lib/python2.7/dist-packages/cfgm_common/zkclient.py", line 337, in master_election
self._election.run(func, *args, **kwargs)
File "/usr/lib/python2.7/dist-packages/kazoo/recipe/election.py", line 48, in run
func(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 807, in run_schema_transformer
args.api_server_ip, args.api_server_port, api_server_use_ssl=args.api_server_use_ssl)
File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 322, in __init__
retry_on_error=False)
File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 748, in _request
content = self._request(op, url, data=data, retry_after_authn=True)
File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 773, in _request
raise HttpError(status, content)
HttpError: HTTP Status: 401 Content: Authentication required |
|
