Comment 1 for bug 1463642

When following sequence of operations were done, the rabbimq drainer greenlet died forever. The rabbitmq drainer should keep running i.e. handle catch-all Exception

1. patch oslo for some backports on all controllers
2. restart rabbitmq server
3. restart all nova services
[ contrail-api was NOT restarted]

13.1.0.2 - - [2015-06-09 17:42:49] "GET /virtual-routers?detail=True HTTP/1.1" 401 192 0.022938
ERROR:cs-scale-2:contrail-api:Config:0:__default__ [SYS_ERR]: VncApiError: RabbitMQ connection down
WARNING:cs-scale-2:contrail-api:Config:0:__default__ [SYS_NOTICE]: VncApiError: RabbitMQ connection ESTABLISHED <Connection: amqp://guest@13.1.0.10:5673// at 0x7f085e208610>
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/gevent/greenlet.py", line 327, in run
    result = self._run(*self.args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/cfgm_common/vnc_kombu.py", line 112, in _connection_watch
    self._reconnect()
  File "/usr/lib/python2.7/dist-packages/cfgm_common/vnc_kombu.py", line 99, in _reconnect
    callbacks=[self._subscribe])
  File "/usr/lib/python2.7/dist-packages/kombu/messaging.py", line 357, in __init__
    self.revive(self.channel)
  File "/usr/lib/python2.7/dist-packages/kombu/messaging.py", line 369, in revive
    self.declare()
  File "/usr/lib/python2.7/dist-packages/kombu/messaging.py", line 379, in declare
    queue.declare()
  File "/usr/lib/python2.7/dist-packages/kombu/entity.py", line 505, in declare
    self.queue_declare(nowait, passive=False)
  File "/usr/lib/python2.7/dist-packages/kombu/entity.py", line 531, in queue_declare
    nowait=nowait)
  File "/usr/lib/python2.7/dist-packages/amqp/channel.py", line 1258, in queue_declare
    (50, 11), # Channel.queue_declare_ok
  File "/usr/lib/python2.7/dist-packages/amqp/abstract_channel.py", line 69, in wait
    return self.dispatch_method(method_sig, args, content)
  File "/usr/lib/python2.7/dist-packages/amqp/abstract_channel.py", line 87, in dispatch_method
    return amqp_method(self, args)
  File "/usr/lib/python2.7/dist-packages/amqp/channel.py", line 241, in _close
    reply_code, reply_text, (class_id, method_id), ChannelError,
NotFound: Queue.declare: (404) NOT_FOUND - home node 'rabbit@cs-scale-1-ctrl' of durable queue 'vnc_config.cs-scale-2-9100' in vhost '/' is down or inaccessible
<Greenlet at 0x7f085e2017d0: <bound method VncServerKombuClient._connection_watch of <vnc_cfg_api_server.vnc_cfg_ifmap.VncServerKombuClient object at 0x7f085f3ac490>>> failed with NotFound

WARNING:keystoneclient.middleware.auth_token:Unable to find authentication token in headers
13.1.0.3 - - [2015-06-09 17:46:04] "GET / HTTP/1.1" 401 192 0.000700
13.1.0.3 - - [2015-06-09 17:46:04] "GET / HTTP/1.1" 200 11310 0.017901
13.1.0.3 - - [2015-06-09 17:46:04] "GET /physical-routers?count=False&detail=False HTTP/1.1" 200 132 0.002907
WARNING:keystoneclient.middleware.auth_token:Unable to find authentication token in headers
13.1.0.2 - - [2015-06-09 17:46:07] "GET / HTTP/1.1" 401 192 0.000702
13.1.0.2 - - [2015-06-09 17:46:08] "GET / HTTP/1.1" 200 11310 0.214082