reduce severity of network notfound trace when looked up by dhcp agent

[Armando Migliaccio]

Neutron Server log has a gazillion of these traces:

2013-11-15 00:40:31.639 8016 ERROR neutron.openstack.common.rpc.amqp [-] Exception during message handling
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp Traceback (most recent call last):
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp File "/opt/stack/new/neutron/neutron/openstack/common/rpc/amqp.py", line 438, in _process_data
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp **args)
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp File "/opt/stack/new/neutron/neutron/common/rpc.py", line 44, in dispatch
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp neutron_ctxt, version, method, namespace, **kwargs)
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp File "/opt/stack/new/neutron/neutron/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs)
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp File "/opt/stack/new/neutron/neutron/db/dhcp_rpc_base.py", line 150, in get_dhcp_port
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp network = plugin.get_network(context, network_id)
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 352, in get_network
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp result = super(Ml2Plugin, self).get_network(context, id, None)
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 1013, in get_network
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp network = self._get_network(context, id)
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 252, in _get_network
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp raise q_exc.NetworkNotFound(net_id=id)
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp NetworkNotFound: Network 6f199bbe-75ad-429a-ac7e-9c49bc389be5 could not be found
2013-11-15 00:40:31.639 8016 TRACE neutron.openstack.common.rpc.amqp

These are about the dhcp agent wanting the sync the state between its local representation of the one on the server's. But an unfound network should be tolerated and no exception trace should be reported.

[Eugene Nikanorov (enikanorov)]

Why do you think exception trace should not be logged? Isn't it regulated with logging level?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/56785

[Maru Newby (maru)]

The proposed fix looks reasonable, but I'm wondering if a better approach might be changing deletion semantics of the neutron service. If the agents could know explicitly that a network was deleted (as opposed to 'should be there but is missing'), we could avoid logging such cases altogether.

[Armando Migliaccio (armando-migliaccio)]

I could not agree with you more. That, however, would require quite an extensive work that would span many parts of the code base as well as may have potential impact to the plugins. It sounds more like blueprint material.

Here, I am trying to find a reasonable balance between effort and speed for a fix that's our gate infrastructure desperately needs.

What do you think?

[Maru Newby (maru)]

+1 on being pragmatic. Let's raise the issue in the next meeting and see if we can get consensus on the priority of the more involved solution.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/56785
Committed: http://github.com/openstack/neutron/commit/f1b9ac5a542a3125d757094fccda80c80c6dd420
Submitter: Jenkins
Branch: master

commit f1b9ac5a542a3125d757094fccda80c80c6dd420
Author: armando-migliaccio <email address hidden>
Date: Sat Nov 16 10:07:31 2013 -0800

    Reduce the severity of dhcp related log traces

    Due to timing issues, when the agent synchronizes
    with the server wrt the representation of server's
    and agent's networks, networks may have already
    been deleted from the db. In such circumstances,
    it is more appropriate to emit warnings rather than
    full-blown exceptions as the agent handles these
    circumstances just fine.

    Close-bug: 1251874

    Change-Id: I4b118b6087c7f42226f0d4265d2176c028d99f22

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/60566

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/60566
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3f4c282d0470fbe56b27fbc1681bae34259ba071
Submitter: Jenkins
Branch: stable/havana

commit 3f4c282d0470fbe56b27fbc1681bae34259ba071
Author: armando-migliaccio <email address hidden>
Date: Sat Nov 16 10:07:31 2013 -0800

    Reduce the severity of dhcp related log traces

    Due to timing issues, when the agent synchronizes
    with the server wrt the representation of server's
    and agent's networks, networks may have already
    been deleted from the db. In such circumstances,
    it is more appropriate to emit warnings rather than
    full-blown exceptions as the agent handles these
    circumstances just fine.

    Close-bug: 1251874

    (cherry picked from commit f1b9ac5a542a3125d757094fccda80c80c6dd420)

    Conflicts:

            neutron/agent/dhcp_agent.py

    Change-Id: I4b118b6087c7f42226f0d4265d2176c028d99f22