Exception remote RPC Callback causes DHCP agent to terminate

[Gary Kotton]

The RPC calls should treat exceptions.

gkotton@gkotton-ubuntu12:~/devstack$ sudo python /opt/stack/quantum/bin/quantum-dhcp-agent --config-file /etc/quantum/quantum.conf --config-file=/etc/quantum/dhcp_agent.ini
Traceback (most recent call last):
  File "/opt/stack/quantum/bin/quantum-dhcp-agent", line 20, in <module>
    main()
  File "/opt/stack/quantum/quantum/agent/dhcp_agent.py", line 505, in main
    mgr.run()
  File "/opt/stack/quantum/quantum/agent/dhcp_agent.py", line 73, in run
    self.enable_dhcp_helper(network_id)
  File "/opt/stack/quantum/quantum/agent/dhcp_agent.py", line 103, in enable_dhcp_helper
    network = self.plugin_rpc.get_network_info(network_id)
  File "/opt/stack/quantum/quantum/agent/dhcp_agent.py", line 222, in get_network_info
    topic=self.topic))
  File "/opt/stack/quantum/quantum/openstack/common/rpc/proxy.py", line 80, in call
    return rpc.call(context, self._get_topic(topic), msg, timeout)
  File "/opt/stack/quantum/quantum/openstack/common/rpc/__init__.py", line 102, in call
    return _get_impl().call(cfg.CONF, context, topic, msg, timeout)
  File "/opt/stack/quantum/quantum/openstack/common/rpc/impl_kombu.py", line 712, in call
    rpc_amqp.get_connection_pool(conf, Connection))
  File "/opt/stack/quantum/quantum/openstack/common/rpc/amqp.py", line 368, in call
    rv = list(rv)
  File "/opt/stack/quantum/quantum/openstack/common/rpc/amqp.py", line 336, in __iter__
    raise result
quantum.openstack.common.rpc.common.RemoteError: Remote error: NameError global name 'cause_exception' is not defined
[u'Traceback (most recent call last):\n', u' File "/opt/stack/quantum/quantum/openstack/common/rpc/amqp.py", line 275, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n', u' File "/opt/stack/quantum/quantum/openstack/common/rpc/dispatcher.py", line 145, in dispatch\n return getattr(proxyobj, method)(ctxt, **kwargs)\n', u' File "/opt/stack/quantum/quantum/db/dhcp_rpc_base.py", line 59, in get_network_info\n network[\'ports\'] = plugin.get_ports(context, filters=filters)\n', u' File "/opt/stack/quantum/quantum/db/db_base_plugin_v2.py", line 1202, in get_ports\n cause_exception\n', u"NameError: global name 'cause_exception' is not defined\n"].
gkotton@gkotton-ubuntu12:~/devstack$

[dan wendlandt (danwent)]

gary, mark, can you two coordinate on this fix? assigning to RC1 until proven otherwise.

[Mark McClain (markmcclain)]

Just wanted to further clarify the bug a little bit. The termination only occurs when the agent is starting up. If the exception occurs after during the processing of a notification, the agent will continue to run.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/12582

[Gary Kotton (garyk)]

On 09/07/2012 12:35 PM, dan wendlandt wrote:
> gary, mark, can you two coordinate on this fix? assigning to RC1 until
> proven otherwise.
>
> ** Changed in: quantum
> Milestone: None => folsom-rc1
>
> ** Changed in: quantum
> Status: New => Confirmed
>
Sure, no problem

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/12582
Committed: http://github.com/openstack/quantum/commit/64cd776da0c26f9324d3dc2918c321d4b6465d5e
Submitter: Jenkins
Branch: master

commit 64cd776da0c26f9324d3dc2918c321d4b6465d5e
Author: Mark McClain <email address hidden>
Date: Fri Sep 7 12:39:39 2012 -0400

    Fix dhcp agent rpc exception handling

    fixes bug 1046904

    This patch adds exception handling to log rpc exceptions. Previously,
    the agent would terminate due to uncaught errors during initialization.

    Change-Id: I4835c1616e2ccfc9c42c591e8c7446db50721d01