unwanted lbaas related error logs in q-svc screen

Bug #1288188 reported by Oleg Bondarev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Oleg Bondarev
oslo-incubator
Fix Released
Medium
Unassigned

Bug Description

2014-03-03 09:25:31.621 5910 ERROR root [-] Original exception being dropped: ['Traceback (most recent call last):\n', ' File "/opt/stack/new/neutron/neutron/db/loadbalancer/loadbalancer_db.py", line 206, in _get_resource\n r = self._get_by_id(context, model, id)\n', ' File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 144, in _get_by_id\n return query.filter(model.id == id).one()\n', ' File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2323, in one\n raise orm_exc.NoResultFound("No row was found for one()")\n', 'NoResultFound: No row was found for one()\n']
2014-03-03 09:25:31.622 5910 WARNING neutron.services.loadbalancer.drivers.common.agent_driver_base [req-6949f3b4-f991-48b7-8424-b3bfbca7c822 None] Cannot update status: member d439c879-55f7-400f-b6a8-32753f057b05 not found in the DB, it was probably deleted concurrently

There is no need for error log about original exception being dropped as warning log is enough.
This happens due to using save_and_reraise_exception() in loadbalancer_db code:

    def _get_resource(self, context, model, id):
        try:
            r = self._get_by_id(context, model, id)
        except exc.NoResultFound:
            with excutils.save_and_reraise_exception():
                if issubclass(model, Vip):
                    raise loadbalancer.VipNotFound(vip_id=id)
                elif issubclass(model, Pool):
                    raise loadbalancer.PoolNotFound(pool_id=id)
                elif issubclass(model, Member):
                    raise loadbalancer.MemberNotFound(member_id=id)
                elif issubclass(model, HealthMonitor):
                    raise loadbalancer.HealthMonitorNotFound(monitor_id=id)
        return r

where the whole purpose of exception handler is to reraise proper type of exception.

I think save_and_reraise_exception() was designed for cases when new exceptions raised inside exception handler are not expected.
In this particular case I don't see the reason for using save_and_reraise_exception().

As an option I think a parameter can be added to save_and_reraise_exception() constructor to disable logging.

Revision history for this message
Ben Nemec (bnemec) wrote :

I'm wondering if the correct thing to do here is to make the logging code respect the reraise parameter and then use that to say that the exception handler doesn't want to reraise the original exception. That seems like the behavior we want with reraise anyway.

https://github.com/openstack/oslo-incubator/blob/master/openstack/common/excutils.py#L67

Changed in oslo:
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to oslo-incubator (master)

Reviewed: https://review.openstack.org/80222
Committed: https://git.openstack.org/cgit/openstack/oslo-incubator/commit/?id=33a2cee6a690ecfdb2cecfe8f01b0b1dacb5bd17
Submitter: Jenkins
Branch: master

commit 33a2cee6a690ecfdb2cecfe8f01b0b1dacb5bd17
Author: Oleg Bondarev <email address hidden>
Date: Thu Mar 13 13:32:09 2014 +0400

    save_and_reraise_exception: make logging respect the reraise parameter

    Do not log original exception if reraise is set to False

    Closes-Bug: #1291850
    Related-Bug: #1288188

    Change-Id: Icd5fcba25c2cd549cee70353a7a62d83bfe1255b

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/81480

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/81549

Changed in neutron:
status: New → In Progress
Changed in oslo:
status: Triaged → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.openstack.org/81480
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=98fc828801f370c0561f22d97ffdd1055d4e0663
Submitter: Jenkins
Branch: master

commit 98fc828801f370c0561f22d97ffdd1055d4e0663
Author: Oleg Bondarev <email address hidden>
Date: Wed Mar 19 12:32:47 2014 +0400

    Sync excutils from oslo

    In order to fix undesired error logs in Neutron (bug 1288188)
    fixed save_and_reraise_exception() should be synced from oslo.
    Oslo commit: 33a2cee6a690ecfdb2cecfe8f01b0b1dacb5bd17

    Also sync gettextutils module as excutils depends on it
    Latest commit in oslo: fd33d1eaa039913d8c82b94c511a3eab0c3d5789

    Closes-Bug: #1294537
    Related-Bug: #1288188
    Change-Id: I62ab3e4e22aa000f3a8d1be26ef9e1cfb1714959

Changed in neutron:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/81549
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=6206d555c3cd129a9ea83174ee786e1e15a4c48c
Submitter: Jenkins
Branch: master

commit 6206d555c3cd129a9ea83174ee786e1e15a4c48c
Author: Oleg Bondarev <email address hidden>
Date: Wed Mar 19 18:38:49 2014 +0400

    Fix usage of save_and_reraise_exception

    Set reraise=False for the cases where the purpose of exception
    handler is to reraise proper type of exception

    Change-Id: Id6595ba1e160b9033d519ded16e0fbd2f91ccb5d
    Closes-Bug: #1288188

Akihiro Motoki (amotoki)
Changed in neutron:
importance: Undecided → Medium
milestone: none → icehouse-rc1
Thierry Carrez (ttx)
Changed in neutron:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in oslo:
milestone: none → icehouse-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in oslo:
milestone: icehouse-rc1 → 2014.1
Thierry Carrez (ttx)
Changed in neutron:
milestone: icehouse-rc1 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.