Comment 4 for bug 1461119

I see the following error in contrail-vrouter-nodemgr-stdout.log

----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 46438)
----------------------------------------
RESULT 2
OKREADY

since supervisord didn't expect/recognize the above message in stdout, it stopped sending further notifications to the event listener (nodemgr).

This exception seem to have occurred when the client closed the socket connection before the response is sent for the introspect request. I have pasted the traceback of this exception from contrail-vrouter-nodemgr-stderr.log.

Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 86, in run
    self.finish_response()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 128, in finish_response
    self.write(data)
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 212, in write
    self.send_headers()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 270, in send_headers
    self.send_preamble()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 194, in send_preamble
    'Date: %s\r\n' % format_date_time(time.time())
  File "/usr/lib/python2.7/socket.py", line 324, in write
    self.flush()
  File "/usr/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
  File "/usr/lib/python2.7/dist-packages/gevent/socket.py", line 458, in sendall
    data_sent += self.send(_get_memory(data, data_sent), flags)
  File "/usr/lib/python2.7/dist-packages/gevent/socket.py", line 435, in send
    return sock.send(data, flags)
error: [Errno 32] Broken pipe
127.0.0.1 - - [02/Jun/2015 17:11:06] "GET /Snh_SandeshUVECacheReq?x=NodeStatus HTTP/1.1" 500 59
Traceback (most recent call last):
  File "/usr/lib/python2.7/SocketServer.py", line 295, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.7/SocketServer.py", line 651, in __init__
    self.finish()
  File "/usr/lib/python2.7/SocketServer.py", line 710, in finish
    self.wfile.close()
  File "/usr/lib/python2.7/socket.py", line 279, in close
    self.flush()
  File "/usr/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
  File "/usr/lib/python2.7/dist-packages/gevent/socket.py", line 458, in sendall
    data_sent += self.send(_get_memory(data, data_sent), flags)
  File "/usr/lib/python2.7/dist-packages/gevent/socket.py", line 435, in send
    return sock.send(data, flags)
error: [Errno 32] Broken pipe

The above trace is printed by handle_error() in SocketServer.py

I could recreate the issue with less timeout value (say 0.001) in contrail-status

Fix:
-----

we use wsgiref module for introspect connection (pysandesh/sandesh_http.py)
WSGIServer is derived from BaseHTTPServer.HTTPServer which is SocketServer.TCPServer
To alleviate this issue, we need to override handle_error() function (refer class BaseServer in SocketServer.py)