[3.0-2680~juno ] Contrail-discovery service: Failed to import package "sandesh"

[chhandak]

From contrail-discovery-0-stdout.log
--------------------------------------

127.0.0.1 - - [2015-12-09 14:18:40] "POST /subscribe HTTP/1.1" 200 135 0.008125
127.0.0.1 - - [2015-12-09 14:18:45] "POST /subscribe HTTP/1.1" 200 135 0.008374
127.0.0.1 - - [2015-12-09 14:18:52] "POST /subscribe HTTP/1.1" 200 135 0.010233
12/09/2015 02:19:04 PM [contrail-discovery]: SANDESH: CONNECT TO COLLECTOR: True
12/09/2015 02:19:04 PM [contrail-discovery]: Failed to import package "sandesh"
12/09/2015 02:19:04 PM [contrail-discovery]: Failed to import package "sandesh"
12/09/2015 02:19:04 PM [contrail-discovery]: SANDESH: Logging: LEVEL: [SYS_INFO] -> [SYS_NOTICE]
Bottle v0.12.0 server starting up (using GeventServer())...
Listening on http://0.0.0.0:9110/
Hit Ctrl-C to quit.

[chhandak (chhandak)]

Logs saved at http://mayamruga.englab.juniper.net/bugs/1524517

[Sundaresan Rajangam (srajanga)]

discovery was not flapping due to import issue. api-server was exiting continuously as it was unable to communicate with keystone.

backtrace from contrail-api-0-stdout.log:

Traceback (most recent call last):
  File "/usr/bin/contrail-api", line 9, in <module>
    load_entry_point('vnc-cfg-api-server==0.1dev', 'console_scripts', 'contrail-api')()
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 2822, in server_main
    main()
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 2781, in main
    vnc_api_server = VncApiServer(args_str)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1392, in __init__
    self._pipe_start_app = auth_svc.get_middleware_app()
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_auth_keystone.py", line 218, in get_middleware_app
    self._auth_token = auth_middleware.get_admin_token()
  File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 761, in get_admin_token
    self.admin_token_expiry) = self._request_admin_token()
  File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 862, in _request_admin_token
    body=params)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 829, in _json_request
    response = self._http_request(method, path, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 796, in _http_request
    raise NetworkError('Unable to communicate with keystone')
NetworkError: Unable to communicate with keystone

As per the following rule configured in /etc/contrail/supervisord_config_files/contrail-config.rules

{"processname": "contrail-api", "process_state": "PROCESS_STATE_EXITED", "action": "sudo service supervisor-config restart"},

supervisor-config gets restarted when contrail-api exits.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15732
Submitter: Sundaresan Rajangam (<email address hidden>)

[Sundaresan Rajangam (srajanga)]

Chhandak, can you please update the synopsis and description to reflect the actual issue - contrail-api exiting due to NetworkError while connecting to keystone.

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15732
Committed: http://github.org/Juniper/contrail-controller/commit/3d7147c7bdcafd351cfa143fd9ea7f0079edb935
Submitter: Zuul
Branch: master

commit 3d7147c7bdcafd351cfa143fd9ea7f0079edb935
Author: Sundaresan Rajangam <email address hidden>
Date: Wed Dec 9 22:39:48 2015 -0800

Specify the full path of the sandesh packages in init_generator()

In the call to init_generator(), sandesh packages list is passed as one
of the parameneters. Presently, the relative path of the package is
being passed ['sandesh']. Therefore, the sandesh library is unable to
import the package and logs error.

Passing the full path of the sandesh package ['discovery.sandesh'] in
the packages list to fix the import error.

Change-Id: I05d120b58a5b8dcf445463d9e1f86a1035702b79
Related-bug: #1524517