Invalid response code from API server

Bug #1723335 reported by Adam Wierzbicki
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
Trunk
Fix Committed
Undecided
Sachin Bansal
OpenContrail
Fix Committed
Undecided
Sachin Bansal

Bug Description

I get the following traceback in service monitor logs:

Traceback (most recent call last):
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/svc_monitor/svc_monitor.py", line 433, in create_service_instance
    self.netns_manager.create_service(st, si)
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/svc_monitor/instance_manager.py", line 780, in create_service
    vm = self._check_create_netns_vm(index, si, st, vm_list[index])
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/svc_monitor/instance_manager.py", line 498, in _check_create_netns_vm
    vm_obj=vm_obj)
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/svc_monitor/instance_manager.py", line 641, in _create_svc_vm_port
    self._vnc_lib.chown(iip_obj.uuid, proj_obj.uuid)
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/vnc_api/vnc_api.py", line 1385, in chown
    data=json.dumps(payload))
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/vnc_api/vnc_api.py", line 855, in _request_server
    retry_after_authn=retry_after_authn, retry_count=retry_count)
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/vnc_api/vnc_api.py", line 960, in _request
    raise HttpError(status, content)
HttpError: HTTP Status: 500 Content: Internal Server Error

At the same time in API server logs I get this error:

Traceback (most recent call last):
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/bottle.py", line 862, in _handle
    return route.call(**args)
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/bottle.py", line 1740, in wrapper
    rv = callback(*a, **ka)
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/vnc_cfg_api_server/context.py", line 122, in wrapper
    return fn(*args, **kwargs)
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1578, in handler_trap_exception
    response = handler(*args, **kwargs)
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1824, in obj_chown_http_post
    obj_fields=['perms2'])
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/vnc_cfg_api_server/vnc_db.py", line 1171, in dbe_read
    obj_type, [obj_id], obj_fields, ret_readonly=ret_readonly)
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/cfgm_common/vnc_cassandra.py", line 496, in wrapper
    return func(*args, **kwargs)
  File "/mnt/xbend/build/debug/container/kube-manager/.venv/lib/python2.7/site-packages/cfgm_common/vnc_cassandra.py", line 847, in object_read
    raise NoIdError(obj_uuids[0])
NoIdError: Unknown id: 9a211a55-0393-480c-b7f0-0adaf0dce223

This seems to be an issue with the API server. If NoIdError occurs during request processing, the return code from the API should be 404, not 500.

If it were 404, the error would have been caught in service monitor.

Changed in opencontrail:
assignee: nobody → Sachin Bansal (sbansal)
Revision history for this message
Adam Wierzbicki (adam.wierzbicki) wrote :

I don't think this is a duplicate.

I think this is an issue with the API server. There should be try-except block around the following lines in vnc_cfg_api_server.py:

try:
    (ok, obj_dict) = self._db_conn.dbe_read(obj_type, obj_uuid,
                                            obj_fields=['perms2'])
except NoIdError as e:
    raise cfgm_common.exceptions.HttpError(404, str(e))

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/36620
Submitter: Adam Wierzbicki (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/36620
Committed: http://github.com/Juniper/contrail-controller/commit/dd395cc766473fefa5fa9517edd29c4c23ceeab6
Submitter: Zuul (<email address hidden>)
Branch: master

commit dd395cc766473fefa5fa9517edd29c4c23ceeab6
Author: Adam Wierzbicki <email address hidden>
Date: Wed Oct 18 10:12:11 2017 +0200

Added NoIdError handling for dbe_read in obj_chown_http_post

Change-Id: I98215c1d48f5c73b9b3d37c365759e2fb303553e
Closes-Bug: #1723335

Sachin Bansal (sbansal)
Changed in opencontrail:
status: New → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.